Investigation assist system and investigation assist method

ABSTRACT

An investigation assist system includes an investigation assist device that acquires captured video of cameras installed at a plurality of intersections respectively; and an operator terminal that is connected to the investigation assist device so as to communicate with the investigation assist device. The investigation assist device detects an occurrence of an event at a first point based on the acquired captured video of the camera, and transmits the captured video of the camera to the operator terminal, the captured video being acquired at a time when the event is detected. The operator terminal records the captured video of the camera transmitted from the investigation assist device, and transmits report contents information based on the report indicating the occurrence of the event to the investigation assist device. The investigation assist device extracts a captured video satisfying the report contents information transmitted from the operator terminal, and transmits the extracted video to the investigation assist device. The operator terminal displays the captured video transmitted from the investigation assist device on the display.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to an investigation assist system and aninvestigation assist method.

2. Description of the Related Art

In the related art, a technique is known in which a plurality of camerasare disposed at predetermined locations on a traveling route of avehicle, and camera image information captured by the respective camerasis displayed on a display device in a terminal device mounted in thevehicle through a network and wireless information exchange device (forexample, refer to JP-A-2007-174016). According to JP-A-2007-174016, auser can obtain a real-time camera image with a large informationamount, based on the camera image information captured by the pluralityof cameras disposed on the traveling route of the vehicle.

In JP-A-2007-174016, the camera image information captured by theplurality of cameras is displayed on the display device in the terminaldevice mounted on the vehicle. Therefore, a user (for example, driver)can check real-time captured images at the locations where therespective cameras are disposed. However, in JP-A-2007-174016, it is notconsidered the method of narrowing down an escape route of a vehiclehaving caused an event such as an incident or accident, when the eventoccurred around the traveling route of the vehicle (for example, anintersection where many people and vehicles come and go). For thisreason, although the technique described in JP-A-2007-174016 is usedwhen the above-described event such as an incident or accident occurred,the system cannot track the escape route of the getaway vehicle, and hasdifficulties in finding the suspect or criminal of the incident oraccident in early stage.

The present disclosure has been made in view of the above problems, andan object thereof is to provide an investigation assist device, aninvestigation assist method, and an investigation assist system whichcan effectively narrow down escape routes of a getaway vehicle andassist finding the suspect or criminal of an incident or accident inearly stage, when the incident or accident occurred around anintersection where many people and vehicles come and go.

SUMMARY OF THE INVENTION

The present disclosure provides an investigation assist system includingan investigation assist device that acquires captured video of camerasinstalled at a plurality of intersections respectively, and an operatorterminal that is connected to the investigation assist device so as tocommunicate with the investigation assist device. The investigationassist device detects an occurrence of an event at a first point basedon the acquired captured video of the camera, and transmits the capturedvideo of the camera to the operator terminal. The captured video isacquired at a time when the event is detected. The operator terminalrecords the captured video of the camera transmitted from theinvestigation assist device, and transmits report contents informationbased on the report indicating the occurrence of the event to theinvestigation assist device. The investigation assist device extracts acaptured video satisfying the report contents information transmittedfrom the operator terminal, and transmits the extracted video to theinvestigation assist device. The operator terminal displays the capturedvideo transmitted from the investigation assist device on the display.

Furthermore, the present disclosure provides an investigation assistmethod using an investigation assist device that acquires captured videoof cameras installed at a plurality of intersections respectively and anoperator terminal that is connected to the investigation assist deviceso as to communicate with the investigation assist device, theinvestigation assist method having steps of detecting an occurrence ofan event at a first point based on the acquired captured video of thecamera, transmitting the captured video of the camera to the operatorterminal, the captured video being acquired at a time when the event isdetected, recording the captured video of the camera transmitted fromthe investigation assist device, transmitting report contentsinformation based on the report indicating the occurrence of the eventto the investigation assist device, extracting a captured videosatisfying the report contents information transmitted from the operatorterminal, transmitting the extracted video to the investigation assistdevice, and displaying the captured video transmitted from theinvestigation assist device on the display.

According to the present disclosure, it is possible to correctly extractand display a related video suitable for the information delivered bythe reporter when an event such as an incident or accident occurred inor around an intersection where many people and vehicles come and go.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of aninvestigation assist system according to embodiments;

FIG. 2 is a view illustrating an example of an installation layout of aplurality of cameras at intersections;

FIG. 3 is a block diagram illustrating an example of internalconfigurations of a master camera and a slave camera according toembodiments;

FIG. 4 is a block diagram illustrating an example of internalconfigurations of a recording server and a retrieval terminal accordingto embodiments;

FIG. 5 is a block diagram illustrating an example of an internalconfiguration of a cloud server according to embodiments;

FIG. 6 is a sequence diagram illustrating an operation procedure exampleof an investigation scenario in the investigation assist systemaccording to the embodiments;

FIG. 7A is a diagram for describing an example of image processing of arecording server or a cloud server according to a first embodiment;

FIG. 7B is a flowchart illustrating a procedure example of the imageprocessing in the recording server or the cloud server according to thefirst embodiment;

FIG. 8 is a sequence diagram illustrating a procedure example of anoperation of inputting report contents through a retrieval terminal andan operation of retrieving an image through a recording server accordingto a second embodiment;

FIG. 9A a diagram for describing an arrangement example of a pluralityof cameras at an intersection and an arrangement example of videoscaptured by the respective cameras and displayed on one screen;

FIG. 9B is a table for describing first and second display methodsaccording to a third embodiment;

FIG. 10A is a view illustrating a display example of images captured bythe plurality of cameras according to the first display method;

FIG. 10B is a view illustrating a display example of images captured bythe plurality of cameras according to the second display method;

FIG. 11A is a flowchart illustrating a procedure example of an operationof narrowing down a getaway vehicle in a recording server or a cloudserver according to a fourth embodiment;

FIG. 11B is a table showing an example of pattern information;

FIG. 11C is a table showing an example of retrieval results before andafter score adjustment;

FIG. 12A is a diagram illustrating a first display example of a capturedvideo displayed on a retrieval terminal according to a fifth embodiment;

FIG. 12B is a diagram illustrating a second display example of acaptured video displayed on the retrieval terminal according to thefifth embodiment;

FIG. 13 is a diagram for describing an outline example of an operationof narrowing down a getaway vehicle through a recording server or acloud server according to a sixth embodiment;

FIG. 14 is a flowchart illustrating a procedure example of the operationof narrowing down the getaway vehicle through the recording server orthe cloud server according to the sixth embodiment;

FIG. 15 is a diagram for describing an outline example of an operationof narrowing down a getaway vehicle in an investigation assist systemaccording to a seventh embodiment;

FIG. 16 is a diagram for describing the outline example of the operationof narrowing down the getaway vehicle in the investigation assist systemaccording to the seventh embodiment;

FIG. 17 illustrates an example of a screen for displaying an escaperoute of the getaway vehicle; and

FIG. 18 is a flowchart illustrating a procedure example of the operationof narrowing down the getaway vehicle in the investigation assist systemaccording to the seventh embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Background to FirstEmbodiment

In JP-A-2007-174016, the camera image information captured by theplurality of cameras is displayed on the display device in the terminaldevice mounted on the vehicle. Therefore, the user (for example, driver)can check real-time captured images at the locations where therespective cameras are disposed. However, in JP-A-2007-174016, it is notconsidered to extract a face image of a person in a vehicle havingcaused an event such as an incident or accident (for example, thesuspect or criminal of the incident or accident), when the eventoccurred around the traveling route of the vehicle (for example, anintersection where many people and vehicles come and go). In particular,when a getaway vehicle appears on a video captured by a camera installedat an intersection, a person in the getaway vehicle may appear dark. Forthis reason, although the technique described in JP-A-2007-174016 isused when the above-described event such as an incident or accidentoccurred, the system cannot rapidly identify the face of the person inthe getaway vehicle, and has difficulties in finding the suspect orcriminal of the incident or accident in early stage.

Therefore, in a first embodiment, an investigation assist device, aninvestigation assist method, and an investigation assist system areexemplified which can extract a face image of a person in a getawayvehicle with high precision and assist finding the suspect or criminalof an incident or accident in early stage, when the incident or accidentoccurred at an intersection where many people and vehicle come and go.

Hereafter, embodiments will be described in detail in which aninvestigation assist device, an investigation assist method, aninvestigation assist system, an operator terminal, and a video displaymethod are specifically disclosed, with reference to the accompanyingdrawings. However, more detailed descriptions than needed may beomitted. For example, the detailed descriptions of known elements or theduplicated descriptions of substantially the same components may beomitted. This is in order to not only avoid unnecessary redundancy ofthe following descriptions, but also promote understanding of thoseskilled in the art. The accompanying drawings and the followingdescriptions are provided to make a person skilled in the art tounderstand the present disclosure, and the subjects of descriptions inclaims are not limited by the drawings and descriptions.

In the following embodiments, a use case will be exemplified in whichwhen an event such as an incident or accident occurred on or around atraveling route of a vehicle (for example, an intersection where manypeople or vehicles come and go), the system assists an investigation ofa policeman by tracking the whereabouts of the vehicle of a suspect orcriminal in real time after the event occurred.

First Embodiment

FIG. 1 is a block diagram illustrating a configuration example of aninvestigation assist system 100 according to embodiments. FIG. 2 is aview illustrating an example of an installation layout of a plurality ofcameras at an intersection INT1. The investigation assist system 100includes a plurality of cameras installed at each intersection, arecording server 50 and a retrieval terminal 90 installed in a policestation, and a cloud server 70 present on the cloud.

In the investigation assist system 100, a plurality of cameras (forexample, cameras 10 to 13) constitute one segment and are installed ateach intersection. In the plurality of cameras in one segment, any onecamera (for example, the camera 10) serves as a master camera, and theother cameras (for example, the cameras 11 to 13) serve as slavecameras. The master camera can communicate with the plurality of slavecameras connected to the master camera, and communicate with therecording server 50 or the cloud server 70. The slave cameras cancommunicate with the master camera connected to the slave cameras. FIG.1 illustrates that the plurality of cameras are installed at theintersection (one segment), but only one master camera may be installed.Furthermore, only a plurality of master cameras may be installed at theintersection without slave cameras.

The cameras 11 to 13 and 21 to 23 serving as slave cameras aremonitoring cameras which can capture subjects at view angles which wererespectively set when the cameras were installed (for example, videosshowing the situations at the intersections), and transmit the capturedvideos to the cameras 10 and 20 serving the master cameras. Each of thecaptured videos may include not only data of the captured video, butalso identification information of the camera having captured the video(an example of camera information), and the same applies hereafter.

The cameras 10 and 20 serving as the master cameras receive the capturedvideos transmitted by the cameras 11 to 13 and 21 to 23 serving as theslave cameras connected to the cameras, respectively. The cameras 10 and20 are monitoring cameras which can capture subjects at view angles setwhen the master cameras were installed (for example, videos showing thesituations at the intersections). The cameras 10 and 20 correlate thecaptured videos thereof with the captured videos transmitted from theslave cameras, and transmit the videos to the recording server 50 or thecloud server 70.

The cameras 10 to 13 are installed to capture the intersection INT1 asillustrated in FIG. 2. The cameras 10 and 11 are installed tosubstantially face each other. Similarly, the cameras 12 and 13 areinstalled to substantially face each other.

The camera 10 is installed on a assist bar mounted perpendicular to apole Pe1 erected in the vertical direction, and disposed adjacent to atraffic signal SN1. The center of the view angle of the camera 10 is setto the central portion of the intersection INT1, and a stop line HLT2around the central portion of the intersection INT1 and a predeterminedarea around the stop line HLT2 are included in the view angle.Therefore, the camera 10 can capture videos of subjects within the setview angle.

The camera 11 is installed on a assist bar mounted perpendicular to apole Pe2 erected in the vertical direction, and disposed adjacent to atraffic signal SN2. The center of the view angle of the camera 11 is setto the central portion of the intersection INT1, and a stop line HLT1around the central portion of the intersection INT1 and a predeterminedarea around the stop line HLT1 are included in the view angle of thecamera 11. Therefore, the camera 11 can capture videos of subjectswithin the set view angle.

The camera 12 is installed on a assist bar mounted perpendicular to apole Pe3 erected in the vertical direction, and disposed adjacent to atraffic signal SN3. The center of the view angle of the camera 12 is setto the central portion of the intersection INT1, and a stop line HLT4around the central portion of intersection INT1 and a predetermined areaaround the stop line HLT4 are included in the view angle of the camera12. Therefore, the camera 12 can capture videos of subjects within theset view angle.

The camera 13 is installed on a assist bar mounted perpendicular to apole Pe4 erected in the vertical direction, and disposed adjacent to atraffic signal SN4. The center of the view angle of the camera 13 is setto the central portion of the intersection INT1, and a stop line HLT3around the central portion of the intersection INT1 and a predeterminedarea around the stop line HLT3 are included in the view angle of thecamera 13. Therefore, the camera 13 can capture videos of subjectswithin the set view angle.

The cameras 20 to 23 illustrated in FIG. 1 may also be installed at anintersection in the same manner as the cameras 10 to 13. The cameras 10to 13 and 20 to 23 are not limited to the installation layoutillustrated in FIG. 2. For example, as illustrated in FIG. 9A, each ofthe cameras may be installed in such a manner that the optical axis ofthe lens thereof faces the optical axis of the lens of the correspondingcamera installed in the diagonal direction. Although described later indetail, the installation layout illustrated in FIG. 9A shows that thecameras are installed at end areas of sidewalks which lead to anintersection INTC0, and mounted on poles PI1 to PI4 erected in thevertical direction with respect to the ground surface.

The recording server 50 (an example of the investigation assist device)is installed in a police station, receives captured videos transmittedfrom cameras installed at intersections under the jurisdiction of thepolice station (for example, master cameras), and stores the receivedvideos in a storage unit 52 (refer to FIG. 4). The stored videos areanalyzed by the recording server 50 based on a request (instruction)from a retrieval terminal 90, when an event such as an incident oraccident occurred, and used for acquiring detailed information on theincident or accident. The recording server 50 may transmit (upload) apart of the captured videos to the cloud server 70 in order to back upthe captured videos. The part of the captured videos may includecaptured videos designated by an operation of a terminal (notillustrated) used by a manager, for example, captured videos for animportant or serious event. The recording server 50 analyzes thereceived videos, acquires tag information, correlates the acquired taginformation with the analyzed videos, and stores the resultant videos inthe storage unit 52. The tag information may indicate information on theface of a person and the type or color of a vehicle in the capturedvideos. The recording server 50 may detect an occurrence of an eventsuch as an incident or accident at or around the intersection throughthe analysis of the captured videos.

The retrieval terminal 90 (an example of the operator terminal) isinstalled in the police station, and used by an official in the policestation (that is, an operator or policeman using the retrieval terminal90). For example, the retrieval terminal 90 may include a laptop ornotebook PC (Personal Computer). When an incident or accident occurred,the operator receives a call (incoming call) from a person (reporter)who reported the occurrence of the incident or accident to the policestation, makes an inquiry about detailed information on the incident oraccident from the reporter, and correctly records various pieces ofinformation acquired through the inquiry as data. The retrieval terminal90 is not limited to the above-described PC, but may include a computerwith a communication function, such as a smart phone, tablet terminaland PDA (Personal Digital Assistant). The retrieval terminal 90 requests(instructs) the recording server 50 or the cloud server 70 to retrieveor analyze the captured videos stored in the recording server 50 or thecloud server 70, and displays the retrieval result or analysis result onan output unit 94 (refer to FIG. 4).

The cloud server 70 is provided on a network such as a cloud, receivescaptured videos transmitted from the recording server 50 or camerasinstalled at intersections under the jurisdiction of police stations(specifically, master cameras), and stores the received videos in thestorage unit 72 (refer to FIG. 4). The stored captured videos areanalyzed by the cloud server 70 based on a request (instruction) fromthe retrieval terminal 90 when an event such as an incident or accidentoccurred, and used for acquiring detailed information on the incident oraccident. The cloud server 70 analyzes the received videos, acquires taginformation, correlates the acquired tag information with the analyzedvideos, and stores the resultant videos in the storage unit 72. The taginformation may indicate information on the faces of people and thetypes or colors of vehicles in the captured videos. The cloud server 70may detect an occurrence of an event such as an incident or accident ator around an intersection through the analysis of the captured videos.

FIG. 1 illustrates that only one recording server 50 and one retrievalterminal 90 are provided in the police station, but a plurality ofrecording servers 50 and retrieval terminals 90 may be provided.Moreover, a plurality of police stations may be included in theinvestigation assist system 100. Similarly, FIG. 1 illustrates that onlyone cloud server 70 is provided on the cloud, but a plurality of cloudservers may be provided.

FIG. 3 is a block diagram illustrating internal configurations of themaster camera and the slave camera according to embodiments. The cameras11 to 13 corresponding to the slave cameras and the camera 10corresponding to the master camera may be connected through a wired LAN(Local Area Network) such as an Intranet NW2, and connected through alocal wireless network (for example, a wireless LAN or WiGig (registeredtrademark)).

Each of the cameras 11 to 13 and 21 to 23 corresponding to the slavecameras includes a capturing unit 41, a processor 42, a recording unit43 and a local communication unit 44. In order to simplify thedescriptions of the slave cameras, the camera 11 will berepresentatively exemplified. In FIG. 3, however, the camera 11 may bereplaced with any one of the cameras 12, 13, 21, 22 and 23.

The capturing unit 41 includes an imaging lens and a solid state imagingdevice such as a CCD (Charge Coupled Device) image sensor or CMOS(Complementary Metal Oxide Semiconductor) image sensor. The capturingunit 41 outputs data of a captured video for subjects to the processor42 at all times while the camera 11 is powered on, the captured videobeing acquired through a capturing operation of the solid state imagingdevice. The capturing unit 41 may include a pan tilt zoom mechanism forchanging the capturing direction or zoom magnification of the camera.

The processor 42 is configured using a CPU (Central Processing Unit),MPU (Micro Processing Unit), DSP (Digital Signal Processor) or FPGA(Field-Programmable Gate Array).

The processor 42 functions as a control unit of the camera 11, andperforms a control process for controlling overall operations of therespective units of the camera 11, a data input/output process among therespective units of the camera 11, a data calculation process and a datastorage process. The processor 42 operates according to a program anddata stored in the memory 45. The processor 42 uses the memory 45 duringoperation, acquires the current time information, and records (stores)the data of the video captured by the capturing unit 41 in the recordingunit 43. Although not illustrated in FIG. 3, the camera 11 may include aGPS (Global Positioning System) receiver. In this case, the camera 11may acquire the current position information from the GPS receiver,correlate the data of the captured video with the position information,and record the resultant data.

The GPS receiver will be briefly described. The GPS receiver receivessatellite signals from a plurality of GPS signal transmitters (forexamples, four navigation satellites), the satellite signals includingthe signal transmission times and position coordinates of the GPS signaltransmitters. The GPS receiver calculates the current positioncoordinate of the master camera or slave camera based on the pluralityof satellite signals and the reception times of the satellite signals.The calculation may be not performed by the GPS receiver, but performedby the processor 32 or 42 to which an output of the GPS receiver isinput. The reception time information may be used for correcting thesystem time of the master camera or slave camera. The system time isused for recording the capturing times of images constituting a capturedvideo, for example.

The processor 42 may control the capturing condition of the capturingunit 41 according to a control command from outside, received by thelocal communication unit 44. For example, when the control command fromoutside commands the processor to change a capturing direction, theprocessor 42 changes the capturing direction during a capturingoperation of the capturing unit 41, according to the control command.For example, when the control command from outside commands theprocessor 42 to change the zoom magnification, the processor 42 changesthe zoom magnification during a capturing operation of the capturingunit 41, according to the control command. For example, when the controlcommand from outside commands the processor 42 to perform a trackingprocess for the designated subject, the processor 42 tracks thedesignated subject using the captured video data recorded in therecording unit 43, according to the control command. When various kindsof control commands are acquired by the processor 42, the processor 42may perform processes corresponding to the respective control commands.

The processor 42 repeatedly transmits the captured video data recordedin the recording unit 43 to the master camera (for example, the camera10) through the local communication unit 44. The repeated transmissionis not limited to transmitting data whenever a predetermined period oftime elapses, but may include transmitting data whenever an irregulartime interval elapses, and transmitting data over a plurality of times.Hereafter, the same applies.

The recording unit 43 may include a semiconductor memory embedded in thecamera 11 (for example, flash memory) or an external memory medium suchas a memory card (for example, SD card), which is not embedded in thecamera 11. The recording unit 43 correlates the data of the capturedvideo generated by the processor 42 with the identification informationof the camera 11 (an example of camera information) or the date and timeinformation during capturing, and records the resultant data. Therecording unit 43 normally pre-buffers and stores data of a videocaptured for a predetermined time, and continuously stores data of avideo captured for a predetermined time (for example, 30 seconds) beforethe current time. When the recording unit 43 is configured as a memorycard, the recording unit 43 may be freely inserted into and removed fromthe casing of the camera 11.

The local communication unit 44 is configured using a communicationcircuit. The local communication unit 44 transmits the data of thecaptured video recorded in the recording unit 43 to the master camera(for example, the camera 10), based on an instruction of the processor42, through short range wireless communication.

The memory 45 is configured using a RAM (Random Access Memory) and ROM(Read Only Memory), for example, and temporarily stores a program ordata required for performing an operation of the camera 11 andinformation or data generated during the operation of the camera 11. TheRAM is a work memory used during an operation of the processor 42, forexample. The ROM stores a program and data for controlling the processor42 in advance. The memory 45 stores identification information foridentifying the camera 11 (for example, serial number) and variouspieces of setting information.

The camera 10 corresponding to the master camera includes a capturingunit 31, a processor 32, a recording unit 33, a wide-area communicationunit 34, a local communication unit 35 and a memory 36. In order tosimplify the descriptions of the master cameras, the camera 10 will berepresentatively exemplified. In FIG. 3, however, the camera 10 may bereplaced with the camera 20.

The capturing unit 31 includes an imaging lens and a solid state imagingdevice such as a CCD image sensor or CMOS image sensor. The capturingunit 31 outputs data of a captured video of subjects to the processor 32at all times while the camera 10 is powered on, the captured video beingacquired through a capturing operation by the solid state imagingdevice. The capturing unit 31 may include a pan tilt zoom mechanism forchanging the capturing direction or zoom magnification of the camera.

The processor 32 is configured using a CPU, MPU, DSP or FPGA, forexample. The processor 32 functions as a control unit of the camera 10,and performs a control process for controlling overall operations of therespective units of the camera 10, a data input/output process among therespective units of the camera 10, a data calculation process and a datastorage process. The processor 32 operates according to a program anddata stored in the memory 36. The processor 32 uses the memory 36 duringoperation, acquires the current time information, or records data of avideo captured by the capturing unit 31 or data of captured videostransmitted from the slave cameras (for example, cameras 11 to 13) intothe recording unit 33. Although not illustrated in FIG. 3, the camera 10may have a GPS receiver. In this case, the camera 10 may acquire thecurrent position information from the GPS receiver.

The processor 32 may control the capturing condition of the capturingunit 31 according to a control command from outside, received by thelocal communication unit 35. For example, when the control command fromoutside commands the processor 32 to change a capturing direction, theprocessor 32 changes the capturing direction during a capturingoperation of the capturing unit 31, according to the control command.For example, when the control command from outside commands theprocessor 32 to change the zoom magnification, the processor 32 changesthe zoom magnification during a capturing operation of the capturingunit 31, according to the control command. For example, when the controlcommand from outside commands the processor to perform a trackingprocess for a designated subject, the processor 32 tracks the designatedsubject using the captured video data recorded in the recording unit 33,according to the control command. When various kinds of control commandsare acquired by the processor 32, the processor 32 may perform processescorresponding to the respective control commands.

The processor 32 repeatedly transmits the captured video data recordedin the recording unit 33 to the recording server 50 or the cloud server70 through the wide-area communication unit 34 and the network NW1 (forexample, Internet).

The recording unit 33 may include a semiconductor memory embedded in thecamera 10 (for example, flash memory) or an external memory medium suchas a memory card (for example, SD card), which is not embedded in thecamera 10. The recording unit 33 may correlate the captured video datagenerated by the processor 32 with the identification information of thecamera 10 (an example of the camera information) or the date and timeinformation during capturing, and record the resultant data.Furthermore, the recording unit 33 may correlate the captured video datatransmitted from the slave cameras (for example, the cameras 11 to 13)with the identification information of the slave cameras (an example ofcamera information) or the date and time information during capturing,and record the resultant data. The recording unit 33 normallypre-buffers and stores data of a video captured for a predeterminedtime, and continuously stores data of a video captured for apredetermined time (for example, 30 seconds) before the current time.When the recording unit 33 is configured as a memory card, the recordingunit may be freely inserted into and removed from the casing of thecamera 10.

The wide-area communication unit 34 is configured using a communicationcircuit. The wide-area communication unit 34 transmits the capturedvideo data recorded in the recording unit 33 to the recording server 50or the cloud server 70 through the wired network NW1 such as theInternet, based on an instruction of the processor 32. The wide-areacommunication unit 34 may receive a control command of the camera,transmitted from the outside (for example, the recording server 50 orthe cloud server 70), or transmit status information of the camera tothe outside (for example, the recording server 50 or the cloud server70).

The local communication unit 35 is configured using a communicationcircuit. The local communication unit 35 may transmit the controlcommand of the camera, received by the wide-area communication unit 34,to the slave cameras (for example, the cameras 11 to 13) through shortrange wireless communication, for example, or receive data of capturedvideos transmitted from the respective slave cameras (for example, thecameras 11 to 13).

The memory 36 is configured using a RAM and ROM, for example, andtemporarily stores a program or data required for performing anoperation of the camera 10 and information or data generated during theoperation of the camera 10. The RAM is a work memory used during anoperation of the processor 32, for example. The ROM stores a program anddata for controlling the processor 32 in advance. The memory 36 storesidentification information for identifying the camera 10 (for example,serial number) and various pieces of setting information.

FIG. 4 is a block diagram illustrating internal configurations of therecording server 50 and the retrieval terminal 90 according to theembodiments. The recording server 50 and the retrieval terminal 90 maybe connected through the Intranet such as a wired LAN provided in thepolice station, or connected through a local wireless network (forexample, wireless LAN).

The recording server 50 includes a communication unit 51, a storage unit52, a video retrieval unit 53, a video analysis unit 54, a tagassignment unit 55, a pattern extraction unit 56 and a memory 57. Thevideo retrieval unit 53, the video analysis unit 54, the tag assignmentunit 55 and the pattern extraction unit 56 are configured as processorssuch as a CPU, MPU, DSP and FPGA, for example. The pattern extractionunit 56 may not be included in the recording server 50 according to thefirst embodiment.

The communication unit 51 is configured using a communication circuit.The communication unit 51 communicates with a master camera (forexample, the camera 10 or 20) connected through the network NW1 such asthe Internet, and receives a captured video transmitted from the mastercamera (that is, a video showing the situation at the intersection). Thecommunication unit 51 communicates with the retrieval terminal 90through a network such as the Intranet, provided in the police station,receives a request (instruction) transmitted from the retrieval terminal90, or transmits a response to the request (instruction). Thecommunication unit 51 transmits a part of the data of the capturedvideo, stored in the storage unit 52, to the cloud server 70.

The storage unit 52 is configured as a hard disk drive (HDD) or solidstate driver (SSD). The storage unit 52 correlates data of a capturedvideo, transmitted from the master camera (for example, the camera 10 or20), with the identification information of the camera having capturedthe video (an example of camera information) or the date and timeinformation during capturing, and then records the resultant data. Thestorage unit 52 also records road map information including a pluralityof intersections, for example, records updated road map informationwhenever the road map information is updated by a new construction ofroad. The storage unit 52 records intersection camera installation dataindicating the corresponding relation between one or more camerasinstalled at each intersection and the intersection. The intersectioncamera installation data may be correlated with the identificationinformation of the intersection and the identification information ofthe cameras. In the following descriptions, the same applies. Therefore,the storage unit 52 correlates the captured video data of the cameraswith the camera information and the intersection information, andrecords the resultant data.

The video retrieval unit 53 retrieves captured video data satisfying aretrieval key among the captured video data recorded in the storage unit52, based on a retrieval request (retrieval instruction) transmittedfrom the retrieval terminal 90 and containing the retrieval key, andtransmits data of the retrieval result through the communication unit 51such that the data are displayed on the output unit 94 of the retrievalterminal 90.

The video analysis unit 54 analyzes the captured video data recorded inthe storage unit 52, and extracts and acquires information on a subject(for example, a person or vehicle) appearing in the captured video. Thevideo analysis unit 54 may acquire information on subjects, and transmitthe acquired information and the captured video data to the tagassignment unit 55 or record the acquired information and the capturedvideo data, which is a video analysis target, in the storage unit 52.The information on subjects may include information on the type, coloror number plate of a vehicle (for example, a getaway vehicle havingcaused an incident or accident), information capable of specifying aperson in the vehicle, or information on the number of people in thevehicle.

The tag assignment unit 55 correlates the information on the videoanalysis result transmitted from the video analysis unit 54 (hereafter,referred to as ‘tag information’) with the captured video data set to avideo analysis target by the video analysis unit 54, and records theresultant data in the storage unit 52. When assigning the taginformation to the captured video data, the tag assignment unit 55 alsocorrelates the captured video data with the date and time information ofthe captured video set to the video analysis target by the videoanalysis unit 54 and the identification information of the camera havingcaptured the video, and records the resultant data in the storage unit52. Accordingly, the recording server 50 can clearly determine thelocation of the intersection where the video was captured, the date andtime information of the captured video, and the tag information assignedto the captured video.

The pattern extraction unit 56 determines whether vehicles have the samebehavior patterns when routinely passing the intersection, using the taginformation and the captured video data which are stored in the storageunit 52. When determining that the behavior patterns are present, thepattern extraction unit 56 records (stores) information on the behaviorpatterns as pattern information in the storage unit 52. For example,based on a histogram (frequency) of information on the dates and timesat which each vehicle has passed the intersection, for the number of thenumber plate of the vehicle, the pattern extraction unit 56 extractsinformation on the date and time at which the peak of the histogram wasacquired, as the pattern information.

The memory 57 is configured using a RAM and ROM, for example, andtemporarily stores a program or data required for performing anoperation of the recording server 50 and information or data generatedduring the operation of the recording server 50. The RAM is a workmemory used during an operation of a processor PRC1, for example. TheROM stores a program and data for controlling the processor PRC1 inadvance. The memory 57 stores identification information for identifyingthe recording server 50 (for example, serial number) and various piecesof setting information.

The retrieval terminal 90 includes an operation unit 91, a processor 92,a communication unit 93, an output unit 94 and a memory 95. Theretrieval terminal 90 is used by an official in the police station, thatis, an operator working as a policeman. When a witness to an event suchas an incident or accident made a call to report the occurrence of theevent, the operator responds to the call with a headset HDS worn on hishead. The headset HDS is connected to the retrieval terminal 90, andcollects voice generated by the operator or outputs voice of thereporter, transmitted through a telephone (not illustrated) throughwhich the incoming call was made.

The operation unit 91 is an UI (User Interface) for detecting an inputoperation of the operator, and may include a mouse or keyboard. Theoperation unit 91 outputs a signal based on the input operation of theoperator to the processor 92. When the operator wants to check thecaptured video of the intersection at the date and time that theoperator wants to investigate, the operation unit 91 receives an inputof a retrieval key containing the date and time information and theintersection information (for example, the location information of theintersection). Furthermore, when the operator wants to check a capturedvideo of a vehicle (for example, a getaway vehicle) at the date and timethat the operator wants to investigate, the operation unit 91 receivesan input of a retrieval key containing the date and time information andvehicle information (for example, the type or color of the vehicle).

The processor 92 is configured using a CPU, MPU, DSP or FPGA, forexample, functions as a control unit of the retrieval terminal 90,performs a control process of controlling overall operations of therespective units of the retrieval terminal 90, a data input/outputprocess among the respective units of the retrieval terminal 90, a datacalculation process and a data storage process. The processor 92operates according to a program and data stored in the memory 95. Theprocessor 92 uses the memory 95 during operation, and acquires thecurrent time information or displays retrieval result data for variouscaptured videos on the output unit 94, the retrieval result data beingtransmitted from the recording server 50 or the cloud server 70. Inresponse to the input of the retrieval key transmitted from theoperation unit 91, the processor 92 generates a retrieval request(retrieval instruction) including the retrieval key and transmits theretrieval request (retrieval instruction) to the recording server 50 orthe cloud server 70 through the communication unit 93.

The communication unit 93 is configured using a communication circuit.The communication unit 93 communicates with the cloud server 70connected through a network NW3 such as the Internet, and receivesvarious captured videos transmitted from the cloud server 70 (forexample, a captured video requested by the retrieval terminal 90). Thecommunication unit 93 communicates with the recording server 50 througha network such as the Intranet, provided in the police state, andtransmits a retrieval request (instruction) for various captured videosto the recording server 50), or receives a response to the request(instruction). The various captured videos may include a captured videoof a vehicle or intersection which the operator wants to investigate.

The output unit 94 is configured using a display such as an LCD (LiquidCrystal Display) or organic EL (Electroluminescence), for example, anddisplays various captured video data transmitted from the processor 92.The output unit 94 may also be configured as a speaker, for example, andoutput a voice signal sent from the processor 92 (for example, apredetermined warning sound generated when a report call was receivedfrom a telephone (not illustrated).

The memory 95 is configured using a RAM and ROM, for example, andtemporarily stores a program or data required for performing anoperation of the retrieval terminal 90 and information or data generatedduring the operation of the retrieval terminal 90. The RAM is a workmemory used during an operation of the processor 92, for example. TheROM stores a program and data for controlling the processor 92 inadvance. The memory 95 stores identification information for identifyingthe retrieval terminal 90 (for example, serial number) and variouspieces of setting information.

FIG. 5 is a block diagram illustrating an internal configuration of thecloud server 70 according to the embodiments. The cloud server 70 isconnected so as to communicate with the recording server 50 through thenetwork NW1 such as the Internet, and connected so as to communicatewith the retrieval terminal 90 through the network NW3 such as theInternet.

The cloud server 70 includes a communication unit 71, a storage unit 72,a video retrieval unit 73, a video analysis unit 74, a tag assignmentunit 75, a pattern extraction unit 76 and a memory 77. The videoretrieval unit 73, the video analysis unit 74, the tag assignment unit75 and the pattern extraction unit 76 are configured as processors suchas a CPU, MPU, DSP and FPGA, for example. The pattern extraction unit 76may not be included in the cloud server 70 according to the firstembodiment.

The communication unit 71 is configured using a communication circuit.The communication unit 71 performs communication with the recordingserver 50 connected through the network NW1 such as the Internet, andreceives a captured video transmitted from the recording server 50. Thecaptured video may include a captured video designated through anoperation of a terminal (not illustrated) used by a manager, forexample, a captured video of an important or serious incident. Thecommunication unit 71 performs communication with the retrieval terminal90 through the network NW3 such as the Internet, and receives a request(instruction) transmitted from the retrieval terminal 90, or transmits aresponse to the request (instruction).

The storage unit 72 is configured using a HDD or SSD, for example. Thestorage unit 72 correlates captured video data transmitted from themaster camera (for example, the camera 10 or 20) or the recording server50 with the identification information of the camera having captured thevideo (an example of camera information) or the date and timeinformation during capturing, and records the resultant data. Thestorage unit 72 may also record road map information including aplurality of intersections, or record updated road map informationwhenever the road map information is updated by a new construction ofroad. The storage unit 72 records intersection camera installation dataindicating the corresponding relation between one or more camerasinstalled at each intersection and the intersection. Therefore, thestorage unit 72 correlates the captured video data of the cameras withthe camera information and the intersection information, and records theresultant data.

Based on a retrieval request (retrieval instruction) transmitted fromthe retrieval terminal 90 and containing a retrieval key, the videoretrieval unit 73 retrieves captured video data satisfying the retrievalkey among the captured video data recorded in the storage unit 72, andtransmits data of the retrieval result through the communication unit 51such that the data are displayed on the output unit 94 of the retrievalterminal 90.

The video analysis unit 74 analyzes the captured video data recorded inthe storage unit 72, and extracts and acquires information on a subject(for example, a person or vehicle) appearing in the captured video. Thevideo analysis unit 74 may acquire information on subjects, and transmitthe acquired information and the captured video data set to the videoanalysis target to the tag assignment unit 75 or record the acquiredinformation and the captured video data in the storage unit 72. Theinformation on the subjects may include information on the type or colorof a vehicle (for example, a getaway vehicle having caused an incidentor accident), or information capable of specifying a person in thevehicle.

The tag assignment unit 75 correlates the information (tag information)on the video analysis result transmitted from the video analysis unit 74with the captured video data set to the video analysis target by thevideo analysis unit 74, and records the resultant data in the storageunit 72. When assigning the tag information to the captured video data,the tag assignment unit 75 also correlates the captured video data withthe date and time information of the captured video set to the videoanalysis target by the video analysis unit 74 or the identificationinformation of the camera having captured the video, and records theresultant in the storage unit 72. Accordingly, the recording server 70can clearly determine the location of an intersection where a video wascaptured, the date and time information of the captured video, and taginformation assigned to the captured video.

The pattern extraction unit 76 determines whether vehicles have the samebehavior patterns when routinely passing an intersection, using the taginformation and the captured video data which are recorded in thestorage unit 72. When determining that the behavior patterns arepresent, the pattern extraction unit 76 records (stores) information onthe behavior patterns as pattern information in the storage unit 72. Forexample, based on a histogram (frequency) of information on the datesand times at which each vehicle has passed the intersection, for thenumber of the number plate of the vehicle, the pattern extraction unit76 extracts information on the date and time at which the peak of thehistogram was acquired, as the pattern information.

The memory 77 is configured using a RAM and ROM, for example, andtemporarily stores a program or data required for performing anoperation of the cloud server 70 and information or data generatedduring the operation of the cloud server 70. The RAM is a work memoryused during an operation of a processor PRC2, for example. The ROMstores a program and data for controlling the processor PRC2 in advance.The memory 77 stores identification information for identifying thecloud server 70 (for example, serial number) and various pieces ofsetting information.

FIG. 6 is a sequence diagram illustrating an operation procedure of aninvestigation scenario in the investigation assist system 100 accordingto the embodiments. The investigation scenario illustrated in FIG. 6shows an example of an operation procedure (use case) which theinvestigation assist system 100 performs when tracking the whereaboutsof a vehicle driven by a suspect or criminal in real time after an eventsuch as an incident or accident occurred in or around an intersection,for example.

In FIG. 6, a camera may be set to a master camera (for example, thecamera 10 or 20 of FIG. 1) or a slave camera (for example, the slavecamera 11, 12, 13, 21, 22 or 23 of FIG. 1). However, a cameratransmitting a captured video to a server at step Sc3 is the mastercamera. The server is the recording server 50 or the cloud server 70constituting the investigation assist system 100. The police station mayindicate various office devices installed in the police station (forexample, the retrieval terminal 90, a telephone (not illustrated) and aheadset HDS).

In FIG. 6, the camera is installed at a predetermined position of theintersection so as to satisfy a predetermined view angle at step Sc1.For example, the camera is installed on the pole of a traffic signal atthe intersection or the traffic signal. When the installation of thecamera is ended, the camera starts capturing at step Sc2, and recordsthe captured video in the memory (for example, the memory 45 or 36). Thecamera transmits data of a video acquired through the capturingoperation started at step Sc2 to the server (for example, the recordingserver 50) at step Sc3.

When receiving the captured video data transmitted from the camera atstep Sc3, the server records (stores) the captured video data in thestorage unit (for example, the storage unit 52 or 72) at step Sc4. Theserver performs video analysis using the recorded captured video data atstep Sc5. The video analysis may be performed by the server whenever thecaptured video data are recorded in the storage unit (for example, thestorage unit 52 or 72). However, the timing of the video analysis is notlimited to the recording timing, but the video analysis may be performedwhenever an analysis request (analysis command) transmitted from theretrieval terminal 90 is received by the server, or performed at eachpredetermined interval. The server can detect an occurrence of an eventsuch as an incident or accident in or around the intersection throughthe analysis of the captured video.

After performing the video analysis at step Sc5, the server correlatestag information acquired through the video analysis with the capturedvideo data, and records (stores) the resultant data in the storage unit(for example, the storage unit 52 or 72) at step Sc6.

Here, suppose that an accident occurred due to a driving error such as adriver's negligence in keeping eyes forward and the driver got away fromthe accident site, at step Sc7.

The server detects the occurrence of the accident through the videoanalysis of step Sc6 at step Sc8, and reports the occurrence of theaccident to the search terminal 90 in the police station through thenetwork NW1 such as the Internet at step Sc9. Moreover, suppose that awitness having witnessed the occurrence of the accident at step Sc7 madea call to the police station at step Sc10, in order to report theoccurrence of the accident to the police station. The call for reportingthe occurrence of the accident at step Sc10 causes an incoming call inthe police station.

When an operator in the police station confirms (receives) the incomingcall of the report and starts to responds to the call at step Sc11, theoperator starts to hear various pieces of information about the accidentfrom the witness having reported the accident, and inputs various piecesof information acquired through the hearing as data while using theretrieval terminal 90, at step Sc12.

When hearing the occurrence site or the date and time information of theaccident, for example, the operator inputs the occurrence site or thedate and time information as a retrieval key to the retrieval terminal90. Based on an operation of the operator who wants to check a capturedvideo of the accident site at step Sc13, the retrieval terminal 90transmits a retrieval request (retrieval instruction) containing theinput retrieval key to the server (for example, the recording server 50or the cloud server 70) at step Sc14.

When receiving the retrieval request (request instruction) containingthe retrieval key at step Sc14, the server retrieves captured video datasatisfying the retrieval key among the captured video data recorded(stored) in the storage unit (for example, the storage unit 52 or 72) atstep Sc15. The server transmits the retrieval result to the retrievalterminal 90 at step Sc16, the retrieval result including the capturedvideo data retrieved at step Sc15.

The retrieval terminal 90 receives the retrieval result transmitted fromthe server at step Sc16, and displays the retrieval result on the outputunit 94. Accordingly, the operator can check a captured video on theoutput unit 94, the captured video satisfying the various pieces ofinformation heard from the witness having reported the occurrence of theaccident at step Sc10 (for example, the accident site or the date andtime information), and extract information on the criminal having causedthe accident through visual confirmation of the captured video displayedon the output unit 94 at step Sc17. The operator further includesinformation as the retrieval key, the information being additionallyheard about the accident from the witness having reported the accident,and transmits a retrieval request (retrieval command) including theretrieval key to the server (for example, the recording server 50 or thecloud server 70) at step Sc14. In this way, steps Sc14 to Sc16 arerepeated. Therefore, the operator using the retrieval terminal 90 cantrack the vehicle driven by the criminal having caused the accident atstep Sc18.

The retrieval terminal 90 generates a control command (operationcommand) for changing the capturing condition of the camera (forexample, the capturing direction or zoom magnification) according to anoperation of the operator, and transmits the control command to thecorresponding camera through the server at step Sc19. The operator usingthe retrieval terminal 90 prepares the deployment of a patrol car inorder to dispatch policemen to the accident site, for example, at stepSc20.

FIG. 7A is a diagram for describing an image processing operation of therecording server 50 or the cloud server 70 according to the firstembodiment. FIG. 7B is a flowchart illustrating a procedure of the imageprocessing operation in the recording server 50 or the cloud server 70according to the first embodiment. The procedure of the image processingoperation in FIG. 7B corresponds to step Sc5 of the investigationscenario in FIG. 6. In the descriptions of FIG. 7B with reference toFIG. 7A, the image processing operation is performed by the recordingserver 50. However, the image processing operation may be performed bythe cloud server 70.

In FIG. 7B, the video analysis unit 54 constituting the processor PRC1of the recording server 50 reads captured video data recorded (stored)in the storage unit 52 and analyzes the read video data, therebydetecting a vehicle and a window frame FRG1 of the vehicle which appearin various images IMG1 constituting the captured video, at step S1. Forexample, the video analysis unit 54 recognizes the position of thenumber plate of the vehicle, using a publicly technique for recognizingthe number plate of a vehicle, and determines that a predetermined areaformed at a predetermined distance upward from the position is a windowframe. The video analysis unit 54 may recognize a vehicle using apublicly technique for detecting a moving subject, and determine that aspecific position of the vehicle is a window frame.

The video analysis unit 54 detects the brightness of the window frameFRG1 detected at step S1, and adjusts the brightness of the entire imageIMG1 such that the window frame FRG1 becomes bright or a predeterminedvalue or more of brightness can be acquired, at step S2. The videoanalysis unit 54 may adjust the brightness of the entire image IMG1 atstep S2, or adjust only the brightness of the detected window frameFRG1.

The video analysis unit 54 detects the window frame FRG2 in the imageIMG2 adjusted at step S2, and detects a face image FC1 of a person inthe vehicle (for example, the criminal having caused the incident oraccident), using an image within the window frame FRG2. The videoanalysis unit 54 cuts out the detected face image FC1 from the imageIMG2 and performs face authentication, at step S3. The faceauthentication process is to determine whether the face image FC1detected within the window frame FRG2 of the image IMG2 has a pluralityof feature points appearing in the face of the person, and a publiclytechnique may be used for the face authentication process.

After step S3, the video analysis unit 54 correlates the data of theimage IMG1 constituting the captured video read at step S1 (the originalimage corresponding to the video analysis target) with the face imageFC1 within the window frame FRG2, which was determined to be the face ofthe person as the result of the face authentication at step S3, andrecords (stores) the correlated data in the storage unit 52, at step S4.Furthermore, the video analysis unit 54 may correlate the image IMG1 andthe face image FC1, which are stored in the storage unit 52 at step S4,with the captured video data set to the video analysis target, andrecord (store) the correlated data in the storage unit 52. Moreover, thevideo analysis unit 54 may transmit the image IMG1 and the face imageFC1, which are stored in the storage unit 52 at step S4, as the analysisresult through the communication unit 51, such that the analysis resultis displayed on the output unit 94 of the retrieval terminal 90.Therefore, the operator can see the analysis result displayed on theoutput unit 94, correctly recognize the face image of the criminalsitting in the front seat of the vehicle within the captured video inwhich the face of the criminal is difficult to confirm in most cases,and correctly transfer the features of the criminal face to a policemanheading for the incident or accident site, which makes it possible toimprove the efficiency of the initial investigation by the policeman.

In the above-described investigation assist system 100 according to thefirst embodiment, the recording server 50 or the cloud server 70 (anexample of the investigation assist device) are connected to the camerasinstalled at each of the plurality of intersections so as to communicatewith the cameras, and acquire captured images of the cameras installedat an intersection where an event such as an incident or accidentoccurred, among the plurality of intersections, through thecommunication unit 51 or 71. The recording server 50 or the cloud server70 detects a vehicle involved in the occurrence of the event (forexample, the vehicle having caused the incident or accident) and thewindow frame of the vehicle through the video analysis unit 54 or 74 (anexample of a detection unit), based on the acquired captured images ofthe cameras. The recording server 50 adjusts the brightness of the imagewithin the detected window frame of the vehicle through the videoanalysis unit 54 or 74 (an example of an adjusting unit) such that thebrightness becomes equal to or more than a predetermined value. Therecording server 50 extracts the face image of the person in the vehicleinvolved in the occurrence of the event through the video analysis unit54 or 74 (an example of an extraction unit), based on the adjusted imagewithin the window frame. The video analysis unit 54 or 74 correlates theextracted face image with the acquired camera image, and stores theresultant data in the storage unit 52 or 72.

Accordingly, when an incident or accident occurred at an intersectionwhere many people or vehicles come and go, the recording server 50 orthe cloud server 70 can extract a face image of a person in a getawayvehicle with high precision, and efficiently assist finding the suspector criminal of the incident or accident in early stage.

When the entire brightness of the acquired camera image is equal to ormore than the predetermined value, the recording server 50 or the cloudserver 70 may start detecting the vehicle involved in the occurrence ofthe event and the window frame of the vehicle through the video analysisunit 54 or 74. Accordingly, the recording server 50 or the cloud server70 can detect a vehicle and the window frame of the vehicle, appearingin a captured video of the camera, only during a period of time in whichthe entire brightness of the captured video of the camera is equal to ormore than a predetermined value (for example, a period of time in thedaytime), which makes it possible to improve the efficiency of the videoanalysis.

Background to Second Embodiment

In JP-A-2007-174016, the camera image information captured by theplurality of cameras is displayed on the display device in the terminaldevice mounted on the vehicle. Therefore, a user (for example, driver)can check real-time images at the places where the respective camerasare disposed. However, in JP-A-2007-174016, it is not considered toextract the details of an event such as an occurrence location andinform an operator of the extracted details, when the event such as anincident or accident occurred around a traveling route of a vehicle (forexample, an intersection where many pedestrians or vehicles come andgo). In particular, a reporter of the event such as an incident oraccident may not correctly describe where the reporter is. In this case,an operator in a police station has to ask detailed information such asthe occurrence location of the event. For this reason, although thetechnique described in JP-A-2007-174016 is used when the event such asan incident or accident occurred, the operator within the policestation, having received the report, may require labor and time toacquire detailed information such as the occurrence location of theevent such as an incident or accident, thereby having difficulties infinding the suspect or criminal of the incident or accident in earlystage.

Therefore, the following second embodiment provides an investigationassist system and an investigation assist method which can correctlyextract a related video suitable for information received from areporter, ask the reporter about detailed information, and assistfinding the suspect or criminal of an incident or accident in earlystage, when the incident or accident occurred around an intersectionwhere many people and vehicles come and go.

Second Embodiment

Since the investigation assist system according to the second embodimenthas the same configuration as the investigation assist system 100according to the first embodiment, the same components are representedby like reference numerals, the detailed descriptions thereof aresimplified or omitted, and the following descriptions will be focused ondifferent components.

FIG. 8 is a sequence diagram illustrating the procedure of an operationof inputting report contents through the retrieval terminal 90 and anoperation of retrieving an image through the recording server 50according to the second embodiment. A large portion of the operationprocedure illustrated in FIG. 8 is duplicated with the contents of theinvestigation scenario illustrated in FIG. 6, and performed at steps Sc2to Sc18.

In FIG. 8, a camera may be set to a master camera (for example, thecamera 10 or 20 of FIG. 1) or a slave camera (for example, the slavecamera 11, 12, 13, 21, 22 or 23 of FIG. 1). At step S12, however, acamera transmitting a captured video to a server is the master camera.The server is the recording server 50 or the cloud server 70constituting the investigation assist system 100. The police station mayindicate the retrieval terminal 90 installed in the police station.

In FIG. 8, the camera starts capturing, and records the captured videoin a memory (for example, the memory 45 or 36) at step S11. The cameratransmits the captured video data acquired through the capturingoperation to the server (for example, the recording server 50) at stepS12. The server receives the captured video data transmitted from thecamera at step S12, and records (stores) the captured video data in thestorage unit (for example, the storage unit 52 or 72) at step S13. Theserver performs video analysis using the recorded captured video data.

Here, suppose that an accident occurred due to a driving error such as adriver's negligence in keeping eyes forward, and the driver ran awaywhile driving an accident vehicle TRC1.

The server detects the occurrence of the accident through video analysisat step S14, extracts data of a video related to the accident(accident-related video) through the network NW1 such as the Internet atstep S15, and notify the extracted data of the accident-related video tothe retrieval terminal 90 in the police station at step S16. Theretrieval terminal 90 receives the data of the accident-related video,transmitted from the server at step S16, at step S17. Furthermore,suppose that a witness having witnessed the occurrence of the accidentmade a call to the police station at step S18, in order to report theoccurrence of the accident to the police station. The call for reportingthe occurrence of the accident at step S18 causes an incoming call tothe police station, and an operator using the retrieval terminal 90receives the call of the report at step S19.

When the operator in the police station confirms (receives) the incomingcall of the report and starts to respond to the call, the operatorstarts to hear various pieces of information about the accident from thewitness having reported the accident, and inputs the various pieces ofinformation acquired through the hearing as data while using theretrieval terminal 90, at step S20. The report contents input at stepS20 include a retrieval key used to request the server to retrieve avideo, the retrieval key including the type or color of the accidentvehicle TRC1 or the date and time information of the accident.

When hearing the occurrence site or the date and time information of theaccident, for example, the operator inputs the occurrence site or thedate and time information as a retrieval key to the retrieval terminal90. The retrieval key may be input through a keyboard or the like of theoperation unit 91 by the operator. Furthermore, when the operator has aconversation with the witness through a headset HDS, the processor 92may acquire conversation voice from the headset HDS, recognize theconversation contents, and automatically generate a retrieval key. Theretrieval terminal 90 retrieves the accident-related video received atstep S17 using a retrieval request (retrieval instruction) including theinput retrieval key, and displays the corresponding accident video,based on an operation of the operator who wants to check the capturedvideo of the accident site, at step S21. Therefore, the operator canhave a conversation with the witness while watching the displayedaccident video, and check whether the displayed image is identical tothe accident witnessed by the witness, based on the neighboringbuildings or the accident situation. In this way, even when the witnesscannot correctly describe the current location of the witness, theoperator can check the location of the witness within a short time. Whenthe witness cannot describe the incident or accident site, the operatorinevitably asks information on the incident or accident site. In thepresent embodiment, however, the incident or accident site detected bythe camera is shown as a candidate to the operator. Thus, the operatorcan easily narrow down the site. For example, suppose that an incidentor accident was detected by cameras at two different points (forexample, points AA and BB) during a period of time in which a report wasreceived. When the witness knows the color of the vehicle having causedthe incident or accident but does not know the place where the incidentor accident occurred, since accident images of the points AA and BB havebeen sent to the police station at step S17, the operator checks aretrieval result obtained by a retrieval operation (for example, aretrieval operation using the color of the vehicle as a retrieval key)through the operation unit 91 from an image displayed on the output unit94 of the retrieval terminal 90. Thus, it is possible to improve thepossibility that the location of the witness can be specified to any oneof the points AA and BB.

The retrieval terminal 90 receives an input of more information (reportcontents) of the accident site which the operator has known through thereport from the witness, at step S22. Based on an operation of theoperator who wants to check more captured videos of the accident site,the retrieval terminal 90 transmits a retrieval request (retrievalinstruction) containing the retrieval key input at step S22 to theserver (for example, the recording server 50 or the cloud server 70) atstep S23.

When receiving the retrieval request (retrieval instruction) containingthe retrieval key transmitted at step S23, the server retrieves capturedvideo data satisfying the retrieval key among the captured video datarecorded (stored) in the storage unit (for example, the storage unit 52or 72) at step S24. The server transmits a retrieval result to theretrieval terminal 90 at step S25, the retrieval result including thecaptured video data retrieved at step S24.

The retrieval terminal 90 receives the retrieval result transmitted fromthe server at step S25 as the accident video and displays the accidentvideo on the output unit 94 at step S26, the retrieval result includinga captured video of the intersection, corresponding to the date and timeinformation of the accident. Accordingly, the operator can check theaccident video on the output unit 94, the accident video satisfying thevarious pieces of information acquired from the witness having reportedthe occurrence of the accident at step S18 (for example, the accidentsite or the date and time information of the accident), and extractinformation on the criminal having caused the accident from the accidentvideo displayed on the output unit 94. The operator includes theinformation in the retrieval key at step S22, the information beingadditionally heard about the accident from the witness having reportedthe accident, and transmits a retrieval request (retrieval instruction)including the retrieval key to the server (for example, the recordingserver 50 or the cloud server 70) at step S23. In this way, steps S22 toS26 are repeated. Therefore, the operator using the retrieval terminal90 can track the accident vehicle TRC1 driven by the criminal havingcaused the accident, or visually check the feature information of theaccident vehicle TRC1 in detail. Furthermore, steps S20 and S22 may beperformed at the same time. When the witness could correctly describe anaccident site, steps S20 and S21 may be omitted.

In the investigation assist system 100 according to the secondembodiment, when the recording server 50 or the cloud server 70 (anexample of the investigation assist device) detects an occurrence of anevent such as an incident or accident at a first point (for example, anintersection) based on an acquired captured video of a camera, therecording server 50 or the cloud server 70 transmits the captured videoof the camera to the retrieval terminal 90, the captured video beingacquired at the point of time that the event was detected. The retrievalterminal 90 (an example of an operator terminal) records the capturedvideo of the camera, transmitted from the recording server 50 or thecloud server 70, in the storage unit 52 or 72, and transmits reportcontents information input on the basis of the report indicating theoccurrence of the event to the recording server 50 or the cloud server70, the report contents information indicating a retrieval request(retrieval instruction) including the retrieval key. The recordingserver 50 or the cloud server 70 extracts a captured video satisfyingthe report contents information transmitted from the retrieval terminal90, and transmits the extracted video to the retrieval terminal 90. Theretrieval terminal 90 displays the video received from the recordingserver 50 or the cloud server 70 on the output unit 94.

Accordingly, when an event such as an incident or accident occurred inor around an intersection where many people and vehicles come and go,the recording server 50 or the cloud server 70 can correctly extract anddisplay a related video suitable for the information delivered by thereporter (for example, a live video of a camera at the occurrence siteof the event). Therefore, the retrieval terminal 90 can ask the reporterabout the detailed information, and assist finding the suspect orcriminal of the incident or accident in early stage. In other words,while the live video of the camera at the occurrence site of the eventis displayed on the output unit 94, the operator can ask the witnessmore specific questions about the occurrence of the event.

The retrieval terminal 90 transmits the input report contentsinformation to the recording server 50 or the cloud server 70, wheneverthe report contents information based on the report is input. Therecording server 50 or the cloud server 70 extracts captured video datasatisfying the report contents information transmitted from theretrieval terminal 90, and transmits the extracted video data to theretrieval terminal 90. Accordingly, the retrieval terminal 90 canfurther include information as the retrieval key, the information beingadditionally heard about the accident from the witness having reportedthe accident. Furthermore, the retrieval terminal 90 can narrow down thecaptured video data satisfying the plurality of input retrieval keys,and track the accident vehicle TRC1 driven by the criminal having causedthe accident, or visually check the feature information of the accidentvehicle TRC1 in detail.

Background to Third Embodiment

In JP-A-2007-174016, the camera image information captured by theplurality of cameras is displayed on the display device in the terminaldevice mounted on the vehicle. Therefore, a user (for example, driver)can check real-time images at the locations where the respective camerasare disposed. However, in JP-A-2007-174016, it is not considered tomultilaterally display videos captured by a plurality of camerasinstalled around an intersection, when the event occurred around thetraveling route of the vehicle (for example, an intersection where manypeople and vehicles come and go). In particular, when an event such asan incident or accident occurred, information indicating in whichdirection a vehicle involved in the occurrence of the event (forexample, a getaway vehicle) ran away is important for the initialinvestigation. For this reason, although the technique described inJP-A-2007-174016 is used when the event such as an incident or accidentoccurred, an operator of a police station, having received a report onthe occurrence of the event, cannot acquire information in early stage,the information indicating in which direction the vehicle involved inthe occurrence of the event such as an incident or accident (forexample, getaway vehicle) ran away, and has difficulties in finding thesuspect or criminal of the incident or accident in early stage.

Therefore, the following third embodiment provides an operator terminaland a video display method, which can multilaterally display videoscaptured by a plurality of cameras, enable an operator to visuallyacquire information indicating in which direction a vehicle involved inan occurrence of an incident or accident ran away, and assist findingthe suspect or criminal of the incident or accident in early stage, whenthe incident or accident occurred in an intersection where many peopleand vehicles come and go.

Third Embodiment

Since the investigation assist system according to the third embodimenthas the same configuration as the investigation assist system 100according to the first embodiment, the same components are representedby like reference numerals, the detailed descriptions thereof aresimplified or omitted, and the following descriptions will be focused ondifferent components.

FIG. 9A illustrates that a plurality of cameras C1 to C4 are arranged atan intersection INTC0 and videos captured by the respective cameras C1to C4 are displayed on one screen. FIG. 9B is a table for describingfirst and second display methods according to the third embodiment. FIG.10A illustrates that the images captured by the cameras C1 to C4 aredisplayed according to the first display method. FIG. 10B illustratesthat the images captured by the cameras C1 to C4 are displayed accordingto the second display method. The display examples of the capturedvideos in FIGS. 10A and 10B correspond to a specific example in whichthe retrieval terminal 90 receives the captured videos and then displaysthe videos on the output unit 94 at step Sc16 of the investigationscenario illustrated in FIG. 6.

As illustrated in FIG. 9A, the cameras C1 and C3 are installed to faceeach other across the intersection INTC0, and the cameras C2 and C4 areinstalled to face each other with the intersection INTC0 interposedtherebetween. The cameras C1 to C4 are fixedly installed on poles PI1 toPI4 on which traffic signals are respectively installed.

The retrieval terminal 90 (an example of the operator terminal)according to the third embodiment receives data of videos captured bythe plurality of cameras C1 to C4 from the recording server 50 or thecloud server 70, and displays the captured video data on the output unit94. The cameras C1 to C4 may correspond to the cameras 10 to 13 or thecameras 20 to 23 illustrated in FIG. 1. In any case, the cameras C1 toC4 correspond to a plurality of cameras installed around the sameintersection INTC0. In the third embodiment, when the retrieval terminal90 displays the captured video data on one screen of the output unit 94,the captured video data are displayed on one screen through anarrangement based on the first display method (refer to FIG. 10A) or thesecond display method (refer to FIG. 10B).

First, the first display method will be described.

For example, suppose that the operator wants to check a captured videoin a direction corresponding to a viewpoint YJ1 of FIG. 9A, when videoscaptured by four cameras are displayed on the output unit 94.Furthermore, suppose that the operator performed an operation of settingthe viewpoint YJ1 to an observation direction, using the operation unit91. According to the operation of the operator, the retrieval terminal90 displays captured videos GM1 to GM4 of the cameras C1 to C4 on onescreen in the output unit 94. Specifically, the captured video GM1 ofthe camera C1 (refer to a screen A) is disposed with the largest size,and the captured video GM3 of the camera C3 (refer to a screen C) isdisposed with a smaller size than the captured video GM1 of the cameraC1. Furthermore, the captured video GM2 of the camera C2 (refer to ascreen B) is distorted and positioned at the left side of the capturedvideo GM1 of the camera C1, and the captured video GM4 of the camera C4(refer to a screen D) is distorted and positioned at the right side ofthe captured video GM1 of the camera C1 (refer to right side of FIG.9A).

In the first display method, the above-described screen arrangement ofthe captured videos is set based on the positional relation among thecameras which are actually arranged in accordance with the viewpointYJ1.

Specifically, in accordance with the viewpoint YJ1, the position of thecamera C1 corresponds to a position to which the operator pays the mostattention. Therefore, a display region for the captured video GM1 of thecamera C1 has the largest area. Furthermore, in accordance with theviewpoint JY1, the camera C3 is installed at the opposite side of thecamera C1 with the intersection INTC0 interposed therebetween.Therefore, the captured video GM3 of the camera C3 is disposed above thecaptured video GM1 so as to face the captured video GM1 when thecaptured video GM3 is displayed on the output unit 94. Moreover, whenseen from the viewpoint YJ1, the camera C2 is installed at the left sideof the camera C1 (the left side in FIG. 9A). Therefore, the capturedvideo GM2 of the camera C2 is disposed at the left top of the capturedvideo GM1 when the captured video GM2 is displayed on the output unit94. Finally, when seen from the viewpoint YJ1, the camera C4 isinstalled at the right side of the camera C1 (the right side in FIG.9A). Therefore, the captured video GM4 of the camera C4 is disposed atthe right top of the captured video GM1 when displayed on the outputunit 94.

The captured video GM2 of the camera C2 is distorted and displayed insuch a manner that a portion of the captured video at the left viewangle in the optical axis direction during capturing of the camera C2becomes a lower end portion, and a portion of the captured video at theright view angle in the optical axis direction during capturing of thecamera C2 becomes an upper end portion. The distortion process may beperformed by the processor 92 of the retrieval terminal 90, or performedby the processors PRC1 and PRC2 of the recording server 50 and the cloudserver 70.

The captured video GM4 of the camera C4 is distorted and displayed insuch a manner that a portion of the captured video at the left viewangle in the optical axis direction during capturing of the camera C4becomes a lower end portion, and a portion of the captured video at theright view angle in the optical axis direction during capturing of thecamera C4 becomes an upper end portion. The distortion process may beperformed by the processor 92 of the retrieval terminal 90, or performedby the processors PRC1 and PRC2 of the recording server 50 and the cloudserver 70.

In the first display method as shown in FIG. 9B, the captured video GM1of the camera C1 corresponding to the screen A is displayed without aparticular distortion process. The captured video GM2 of the camera C2corresponding to the screen B is distorted and displayed in theabove-described manner. The captured video GM3 of the camera 3corresponding to the screen C is not subjected to the above-describeddistortion process, but resized to a smaller size than the capturedvideo GM1 and then displayed. The captured video GM4 of the camera C4corresponding to the screen B is distorted and displayed in theabove-described manner.

FIG. 10A illustrates one screen WD1 displayed on the output unit 94 ofthe retrieval terminal 90, the screen WD1 showing videos MOV1 to MOV4captured by the actual cameras C1 to C4 according to the first displaymethod of FIG. 9B. When an operator performs an operation of designatinga subject ESC1 of interest (for example, getaway vehicle) while thesubject ESC1 is displayed on the output unit 94 as illustrated in FIG.10A, the retrieval terminal 90 transmits a video analysis request to therecording server 50 or the cloud server 70 in order to specify thesubject ESC1 designated through the operation. The recording server 50or the cloud server 70 detects the subject ESC1 appearing in thecaptured videos MOV2 to MOV4, and transmits the detection position(coordinate information) as a video analysis result to the retrievalterminal 90, based on the video analysis request transmitted from theretrieval terminal 90. The retrieval terminal 90 may apply apredetermined color frame onto the subject ESC1 in the captured videosMOV2 to MOV4 displayed on the output unit 94, based on the videoanalysis result transmitted from the recording server 50 or the cloudserver 70. Accordingly, when the captured videos of the plurality ofcameras are displayed in accordance with the viewpoint YJ1 to which theoperator pays the most attention, the operator can perform a simpleoperation of designating the subject ESC1 of interest (for example,getaway vehicle), thereby checking the same subject with the color framein the captured videos of the other cameras. Thus, the operator caneasily track the subject.

Next, the second display method will be described. The duplicatedcontents with the descriptions of the first display method are omittedherein.

According to an operation of an operator to designate the viewpoint YJ1,the retrieval terminal 90 displays the captured videos GM1 to GM4 of thecameras C1 to C4 on one screen. Specifically, the captured video GM1 ofthe camera C1 (refer to a screen A) is disposed at the largest size. Thecaptured video GM3 of the camera C3 (refer to a screen C) is distortedin the side-to-side direction (horizontal direction), and disposed at asmaller size than the captured video GM1. The captured video GM2 of thecamera C2 (refer to a screen B) is inverted in the side-to-sidedirection (horizontal direction), and distorted and disposed at the leftside of the captured video GM1 of the camera C1. The captured video GM4of the camera C4 (refer to a screen D) is inverted in the side-to-sidedirection (horizontal direction), and distorted and disposed at theright side of the captured video GM1 of the camera C1. In this way, theretrieval terminal 90 displays the captured videos GM1 to GM4 on theoutput unit 94 (refer to the right side of FIG. 9A).

In the second display method, the screen arrangement of the capturedvideos is set based on the positional relation among the captured videoswhich are displayed when the captured video GM1 (refer to the screen A)is disposed in front of a three-sided mirror in accordance with theviewpoint YJ1.

When seen from the viewpoint YJ1, the camera C2 is installed at the leftside of the camera C1 (on the upper side in FIG. 9A). However, when itis assumed that the captured video GM1 is disposed in front of thethree-sided mirror, the captured video GM2 of the camera C2 is disposedat the right top of the captured video GM1 when displayed on the outputunit 94. When seen from the viewpoint YJ1, the camera C3 is installed atthe right side of the camera C1 (the right side in FIG. 9A). However,when it is assumed that the captured video GM1 is disposed in front ofthe three-sided mirror, the captured video GM4 of the camera C4 isdisposed at the left top of the captured video GM1 when displayed on theoutput unit 94.

The captured video GM2 of the camera C2 is inverted in the side-to-sidedirection (horizontal direction), and distorted and displayed in such amanner that a portion of the captured video at the left view angle inthe optical axis direction during capturing of the camera C2 becomes alower end portion, and a portion of the captured video at the right viewangle in the optical axis direction during capturing of the camera C2becomes an upper end portion.

The captured video GM4 of the camera C4 is inverted in the side-to-sidedirection (horizontal direction), and distorted and displayed in such amanner that a portion of the captured video at the left view angle inthe optical axis direction during capturing of the camera C4 becomes alower end portion, and a portion of the captured video at the right viewangle in the optical axis direction during capturing of the camera C4becomes an upper end portion.

In the second display method as shown in FIG. 9B, the captured video GM1of the camera C1 corresponding to the screen A is displayed without aparticular distortion process. The captured video GM2 of the camera C2corresponding to the screen B is inverted in the side-to-side direction(horizontal direction), and distorted and displayed in theabove-described manner. The captured video GM3 of the camera 3corresponding to the screen C is not subjected to the above-describeddistortion process, but inverted in the side-to-side direction(horizontal direction), resized to a smaller size than the capturedvideo GM1, and then displayed. The captured video GM4 of the camera C4corresponding to the screen D is inverted in the side-to-side direction(horizontal direction), and distorted and displayed in theabove-described manner.

FIG. 10B illustrates one screen WD1 displayed on the output unit 94 ofthe retrieval terminal 90, the screen WC1 showing videos MOV1, MOV2 r,MOV3 r and MOV4 r captured by the actual cameras C1 to C4 according tothe second display method of FIG. 9B. When the operator performs anoperation of designating a subject ESC1 of interest (for example,getaway vehicle) while the subject ESC1 is displayed on the output unit94 as illustrated in FIG. 10B, the retrieval terminal 90 transmits avideo analysis request to the recording server 50 or the cloud server 70in order to specify the subject ESC1 designated through the operation.The recording server 50 or the cloud server 70 detects the subject ESC1appearing in the captured videos MOV2 r to MOV4 r, and transmits thedetection position (coordinate information) as a video analysis resultto the retrieval terminal 90, based on the video analysis requesttransmitted from the retrieval terminal 90. The retrieval terminal 90may apply a predetermined color frame onto the subject ESC1 in thecaptured videos MOV2 r to MOV4 r displayed on the output unit 94, basedon the video analysis result transmitted from the recording server 50 orthe cloud server 70. Accordingly, when the captured videos of theplurality of cameras are disposed and displayed in front of thethree-sided mirror in accordance with the viewpoint YJ1 to which theoperator pays the most attention, the operator can perform a simpleoperation of designating the subject ESC1 of interest (for example,getaway vehicle), thereby checking the same subject with the color framein the captured videos of the other cameras. Thus, the operator caneasily track the subject.

In the above-described investigation assist system 100 according to thethird embodiment, the retrieval terminal 90 (an example of the operatorterminal) is connected so as to communicate with the recording server 50or the cloud server 70 which acquires captured videos of the camerasinstalled at each of the plurality of intersections. The retrievalterminal 90 acquires captured videos of the plurality of cameras C1 toC4 through the communication unit 93 (an example of an acquisition unit)from the recording server 50 or the cloud server 70, the plurality ofcameras C1 to C4 being installed around an intersection where an eventoccurred among the plurality of intersections. The retrieval terminal 90sets display regions for the captured videos of the plurality of cameraswithin one screen through the processor 92 (an example of a settingunit), based on the arrangement condition of the plurality of camerassurrounding the intersection where the event occurred. The retrievalterminal 90 displays the captured videos of the plurality of cameras,arranged on the respective display regions in the set screen, on theoutput unit 94 through the processor 92 (an example of a display controlunit).

Accordingly, when an incident or accident occurred in an intersectionwhere many people or vehicles come and go, the retrieval terminal 90 canmultilaterally display the captured videos of the plurality of camerason the output unit 94, enable the operator to visually acquireinformation indicating in which direction the vehicle involved in theoccurrence of the incident or accident ran away, and assist finding thesuspect or criminal of the incident or accident in early stage.

The processor 92 (an example of the setting unit) of the retrievalterminal 90 sets display regions in which captured videos of a set ofcameras (for example, the cameras C2 and C4) facing each other with theintersection interposed therebetween among the plurality of cameras aredistorted in the reverse direction (refer to the first display method).Accordingly, the retrieval terminal 90 can multilaterally display thecaptured videos in accordance with the viewpoint YJ1 designated by theoperator, compared to when the captured videos GM1 to GM4 of theplurality of cameras are displayed at equal sizes in parallel to eachother. Therefore, the operator can efficiently recognize the capturedvideos of the plurality of cameras.

The processor 92 (an example of a display control unit) of the retrievalterminal 90 inverts captured videos of at least a set of cameras (forexample, the cameras C2 and C4) in the horizontal direction, anddisplays the inverted videos on the output unit 94 (refer to the seconddisplay method). Accordingly, compared to when the captured videos GM1to GM4 of the plurality of cameras are displayed at equal sizes inparallel to each other, the retrieval terminal 90 can multilaterallydisplay various captured videos while the most prominent video in isdisposed in front of the three-sided mirror accordance with theviewpoint YJ1 designated by the operator. Therefore, the operator canefficiently recognize the captured videos of the plurality of cameras.At this time, the user (for example, an operator of the police station)can set any one of the first and second display methods as the method ofdisplaying a captured video on the output unit 94, through an inputoperation using the operation unit 91. The setting information is storedin the memory 95 of the retrieval terminal 90, for example.

Background to Fourth Embodiment

In JP-A-2007-174016, the camera image information captured by theplurality of cameras is displayed on the display device in the terminaldevice mounted on the vehicle. Therefore, a user (for example, driver)can check real-time images at the locations where the respective camerasare disposed. However, in JP-A-2007-174016, it is not considered tonarrow down an escape route of a vehicle having caused an event such asan incident or accident, when the event occurred in a traveling route ofthe vehicle (for example, an intersection where many people and vehiclescome and go). In particular, when a vehicle (getaway vehicle) having runaway from the intersection where the event occurred is tracked, thepossibility that a vehicle passing the intersection during the same timeof day will be the getaway vehicle is considered to be low. Therefore,although the technique described in JP-A-2007-174016 is used when theevent such as an incident or accident occurred, an operator of a policestation, having received a report, needs to retrieve candidates of thegetaway vehicle among all vehicles passing through the intersectionwhere the event occurred, when narrowing down the getaway vehiclecorresponding to a tracking target. Furthermore, the operator requireslabor and time to narrow down the candidates of the getaway vehicle, andhas difficulties in finding the suspect or criminal of the incident oraccident in early stage.

Therefore, the following fourth embodiment provides an investigationassist device, an investigation assist method and an investigationassist system, which can efficiently reduce labor and time required fornarrowing down candidates of a getaway vehicle having run away from anintersection, and assist finding the suspect or criminal of an incidentor accident in early stage, when the incident or accident occurred atthe intersection where many people and vehicle come and go.

Fourth Embodiment

Since the investigation assist system according to the fourth embodimenthas almost the same configuration as the investigation assist system 100according to the first embodiment, the same components are representedby like reference numerals, the detailed descriptions thereof aresimplified or omitted, and the following descriptions will be focused ondifferent components. In the fourth embodiment, the recording server 50and the cloud server 70 have pattern extraction units 56 and 76,respectively.

FIG. 11A is a flowchart illustrating a procedure of an operation ofnarrowing down a getaway vehicle in the recording server 50 or the cloudserver 70 according to the fourth embodiment. FIG. 11B is a tableshowing an example of pattern information PTF1. FIG. 11C is a tableshowing retrieval results before and after score adjustment.

When tracking a vehicle (getaway vehicle) having run away from anintersection where an event such as an incident or accident occurred isconsidered, the possibility that a vehicle passing through theintersection during almost the same time of day will be the getawayvehicle is considered to be low. In the fourth embodiment, the recordingserver 50 or the cloud server 70 determines whether each vehicle has abehavior pattern, using captured video data and tag information whichare recorded (stored) in the storage unit 52 or 72. The behavior patternmay indicate whether the same vehicle routinely passes through the sameintersection at the same time of day. When determining that there is avehicle satisfying the above-described behavior pattern, the recordingserver 50 or the cloud server 70 correlates the date and timeinformation and the feature information of the vehicle satisfying thebehavior pattern with the information of the intersection where thebehavior pattern has been detected, and records (stores) the resultantinformation in the storage unit 52 or 72.

Since an event such as an incident or accident does not periodicallyoccur but suddenly occurs, the possibility that a vehicle satisfying theabove-described behavior pattern will be the vehicle having caused theevent such as an incident or accident is considered to be low.Therefore, when the recording server 50 or the cloud server 70 acquiresa retrieval request containing a retrieval key related to the getawayvehicle from the retrieval terminal 90 through an operation of theoperator, for example, the recording server 50 or the cloud server 70retrieves captured video data of a vehicle satisfying patterninformation among one or more vehicles satisfying the retrieval key. Therecording server 50 or the cloud server 70 decides the display priorityof the captured video of the vehicle having caused the event on theretrieval terminal 90, using the retrieval result indicating that thereis the vehicle satisfying the pattern information among the one or morevehicles satisfying the retrieval key. The recording server 50 or thecloud server 70 displays the captured video of the vehicle which islikely to be the getaway vehicle on the retrieval terminal 90, based onthe decided display priority.

As illustrated in FIG. 11B, pattern information PTF1 may include theidentification information (ID) of the pattern information PTF1, thefeature data of a vehicle, and a pattern (date and time information onwhen the vehicle routinely passes through an intersection), which arecorrelated with one another for each intersection. For example, patterninformation with identification information ID1 shows that the frequencyat which a vehicle with feature data of ‘ABC-123’ passes through theintersection between 9:00 a.m. and 9:30 a.m. on weekdays from Monday toFriday is high. Furthermore, pattern information with identificationinformation ID2 shows that the frequency at which a vehicle with featuredata of JJJ-543′ passes through the intersection between 10:00 a.m. and11:00 a.m. on weekdays from Monday to Thursday is high. Moreover,pattern information with identification information ID3 shows that thefrequency at which a vehicle with feature data of ‘PAN-329’ passesthrough the intersection between 12:00 p.m. and 1:00 p.m. on Saturdayand Sunday is high. The feature data may be set to the number of thenumber plate of the corresponding vehicle or data indicating combinationinformation of the type and color of the vehicle or a feature amount ofcaptured image of the vehicle.

The procedure of the video retrieval operation in FIG. 11A correspondsto step Sc15 of the investigation scenario in FIG. 6. In FIG. 11A, whenthe recording server 50 or the cloud server 70 acquires a retrievalrequest containing a retrieval key related to a getaway vehicle and thedate and time information of an event such as an incident or accidentfrom the retrieval terminal 90 according to an operation of theoperator, the recording server 50 or the cloud server 70 retrievescaptured videos of vehicles through the video retrieval unit 53 or 73,the vehicles satisfying the pattern information PTF1 among one or morevehicles satisfying the retrieval key, at step S31. The video retrievalunit 53 or 73 determines whether to adjust a score indicating if each ofthe vehicles is the vehicle having caused the event as the videoretrieval result, depending on the type of the reported event such as anincident or accident, at step S32. The video retrieval unit 53 or 73transmits data of the captured video of the vehicles to the retrievalterminal 90 such that the captured video data are sequentially displayedon the retrieval terminal 90 in descending order of score (for example,the similarity of a vehicle appearing in the captured videos). The scoreis derived by the video retrieval unit 53 or 73 according to a publiclytechnique, for example. For example, when the reported event is an eventsuch as a robbery case, which suddenly occurs, the video retrieval unit53 or 73 determines that the score of the vehicle satisfying the patterninformation PTF1 needs to be adjusted (Yes at step S32). In this case,the procedure of the recording server 50 or the cloud server 70 proceedsto step S33.

On the other hand, when the event is a traffic accident which may becaused by the vehicle satisfying the pattern information PTF1, the videoretrieval unit 53 or 73 determines that the score of the vehiclesatisfying the pattern information PTF1 does not need to be adjusted (Noat step S32). In this case, the procedure of the recording server 50 orthe cloud server 70 proceeds to step S35.

When the video retrieval unit 53 or 73 determines that there is avehicle satisfying the pattern information PTF1 (or corresponding to thepattern information PTF1) among the captured videos obtained through theretrieval of step S31 (Yes at step S33), the video retrieval unit 53 or73 lowers the score of the vehicle satisfying the pattern informationPTF1 by a predetermined value (for example, 10 points) at step S34. Forexample, when a retrieval result SCR1 of the captured video at step S31is obtained from the video of the intersection at 10:30 on Tuesday asillustrated in FIG. 11C, the video retrieval unit 53 or 73 derived thescore of the feature data ‘JJJ-543’ of the vehicle as ‘85’. However,since it was determined that the vehicle satisfies the patterninformation PTF1 after the derivation, the video retrieval unit 53 or 73derives the score as ‘75’ by lowering the score by the predeterminedvalue (for example, 10 points). That is, the video retrieval unit 53 or73 acquires the retrieval result SCR2 of the captured video as theprocessing result of step S34. In the retrieval result SCR2, the scoreof the feature data ‘LMG-006’ of another vehicle is not adjusted becausethe time of day at which the vehicle frequently appears is differentfrom the feature data of the pattern information PTF1, and the score ofthe feature data ‘PAN-329’ of another vehicle is not adjusted becausethe feature information of the vehicle is not included in the patterninformation PTF1.

Based on the captured video retrieval result SCR2 at step S34 or thecaptured video retrieval result SCR1 at step S31, the video retrievalunit 53 or 73 transmits the captured video data of the vehicles suchthat the captured video data are displayed on the output unit 94 of theretrieval terminal 90 according to the display priority which is decidedbased on the scores (step S35).

In the investigation system 100 according to the fourth embodiment, therecording server 50 and the cloud server 70 are connected to the camerasinstalled at the plurality of intersections so as to communicate withthe cameras. The recording server 50 and the cloud server 70 record thepattern information, for each intersection, having the date and timeinformation and the identification information of the vehicles havingpassed through each of the intersections and the captured videos of thecameras in correlation with the camera information and the intersectioninformation, in the storage units 52 and 72. According to an informationinput containing the date and time information and the intersectioninformation of the event and the feature information of the vehiclehaving caused the event (for example, an input of the operator to theretrieval terminal 90), the recording server 50 or the cloud server 70retrieves a vehicle satisfying the pattern information in the capturedvideos of the cameras in the intersection where the event occurred,through the video retrieval units 53 and 73. The recording server 50 orthe cloud server 70 derives a score indicating whether the vehicle isthe vehicle having caused the event, using the retrieval resultindicating the presence of the vehicle satisfying the patterninformation. Based on the derived score, the recording server 50 or thecloud server 70 decides the display priority of the captured video ofthe vehicle having caused the event through the video retrieval unit 53or 73. The recording server 50 or the cloud server 70 displays thecaptured videos of one or more corresponding vehicles on the retrievalterminal 90, based on the decided priority.

Accordingly, when an incident or accident occurred in an intersectionwhere many people or vehicles come and go, the recording server 50 andthe cloud server 70 can efficiently reduce labor and time required fornarrowing down the getaway vehicle having run away from theintersection, and efficiently assist finding the suspect or criminal ofthe accident or incident in early stage.

When it is determined that there is a vehicle satisfying the patterninformation PTF1, the recording server 50 or the cloud server 70 lowersthe score of the vehicle satisfying the pattern information PTF1 by thepredetermined value, and decides the display priority of the capturedvideo data on the retrieval terminal 90. Accordingly, the recordingserver 50 or the cloud server 70 can lower the possibility that avehicle routinely passing through the same intersection in the same timeof day caused an accident or incident which suddenly occurs, andprecisely extract a captured video of a vehicle which does not satisfythe pattern information PTF1.

When it is determined that there are no vehicles satisfying the patterninformation PTF1, the recording server 50 or the cloud server 70 doesnot adjust the score of the vehicle satisfying the pattern informationPTF1, and decides the display priority of the captured video data on theretrieval terminal 90. Accordingly, considering that even the vehiclesatisfying the pattern information PTF1 may cause an incident oraccident such as a traffic accident, the recording server 50 or thecloud server 70 can extract the captured video of the vehicle havingcaused the event such as an incident or accident with high precision.

The recording server 50 or the cloud server 70 generates the patterninformation PTF1 based on the generation frequency of date and timeinformation at which a vehicle passes through each of the intersections,and records (stores) the generated pattern information PTF1 in thestorage unit 52 or 72. Accordingly, the recording server 50 or the cloudserver 70 can generate information for lowering the possibility that avehicle having caused an incident or accident is a vehicle routinelypassing through the same intersection in the same time of day.Therefore, the recording server 50 or the cloud server 70 canefficiently reduce labor and time required for retrieving a capturedvideo of the vehicle having caused the incident or accident.

Background to Fifth Embodiment

In JP-A-2007-174016, the camera image information captured by theplurality of cameras is displayed on the display device in the terminaldevice mounted on the vehicle. Therefore, a user (for example, driver)can check real-time images at the locations where the respective camerasare disposed. However, in JP-A-2007-174016, it is not considered toacquire and display a traffic situation when an event such as anincident or accident occurred on a traveling route of a vehicle (forexample, an intersection where many people and vehicles come and go). Inparticular, when the event such as an incident or accident occurred, areporter of the event such as an incident or accident may not correctlyremember if a traffic signal facing the vehicle having caused the eventwas a red light or blue light. For this reason, although the techniquedescribed in JP-A-2007-174016 is used when the event such as an incidentor accident occurred, an operator within a police station, havingreceived the report, cannot simply ask the traffic situation at theintersection where the event such as an incident or accident occurred,and has difficulties in correctly recording the situation when theincident or accident occurred.

Therefore, the following fifth embodiment provides an operator terminaland a video display method, which can visually display a trafficsituation around a vehicle having caused an incident or accident andassist an operator to correctly record the situation during the incidentor accident, when the incident or accident occurred in an intersectionwhere many people or vehicles come and go.

Fifth Embodiment

Since the investigation assist system 100 according to the fifthembodiment has the same configuration as the investigation assist system100 according to the first embodiment, the same components arerepresented by like reference numerals, the detailed descriptionsthereof are simplified or omitted, and the following descriptions willbe focused on different components.

FIG. 12A is a diagram illustrating a first display example of a capturedvideo displayed on the retrieval terminal 90 according to the fifthembodiment. FIG. 12B is a diagram illustrating a second display exampleof a captured video displayed on the retrieval terminal 90 according tothe fifth embodiment.

In FIGS. 12A and 12B, screens WD3 and WD4 for captured videos aredisplayed on the output unit 94 of the retrieval terminal 90 at stepSc16 of the scenario illustrated in FIG. 6. When receiving an operator'soperation of inputting the date and time information and intersectioninformation of an event such as incident or accident, the retrievalterminal 90 transmits a retrieval request (retrieval intersection) tothe recording server 50 or the cloud server 70, the retrieval requestcontaining the date and time information and the intersectioninformation as a retrieval key. The retrieval terminal 90 displayscaptured video data on the output unit 94, the captured video data beingtransmitted from the recording server 50 or the cloud server 70.

When receiving an operation of inputting a visual field direction RNG1for the intersection where the event such as an incident or accidentoccurred, with respect to the captured video displayed on the outputunit 94, the retrieval terminal 90 transmits an analysis instruction tothe recording server 50 or the cloud server 70 to analyze a situationindicated by a traffic situation indicator (for example, traffic signal)which is present in the visual field direction RNG1, using theinformation of the input visual field direction RNG1 as a retrieval key.Based on the analysis instruction transmitted from the retrievalterminal 90, the recording server 50 or the cloud server 70 analyzes thesituation indicated by the traffic situation indicator (for example,traffic signal) installed in the input visual field direction RNG1,using captured video data of a camera at the corresponding date and timein the intersection set to a retrieval target.

The recording server 50 or the cloud server 70 transmits the analysisresult of the situation indicated by the traffic situation indicator(for example, traffic signal) in the visual field direction RNG1 and thecaptured video data of the camera in the corresponding intersection tothe retrieval terminal 90. The analysis result may include an image of avehicle present in the visual field direction RNG1 at a point of timecorresponding to the date and time information input to the retrievalterminal 90. The retrieval terminal 90 correlates a traffic signal imageSGN1 and a vehicle image ESC11 with each other using the analysis resulttransmitted from the recording server 50 or the cloud server 70, andoverlaps the traffic signal image SGN1 and the vehicle image ESC11 withthe captured video to display on the output unit 94. The traffic signalimage SGN1 shows a state of the traffic situation indicator (forexample, traffic signal) in the visual field direction RNG1, and thevehicle image ESC11 shows a vehicle in the visual field direction RNG1.The traffic signal image SGN1 may be generated by the recording server50 or the cloud server 70, and generated by the retrieval terminal 90based on the analysis result. The traffic signal image SGN1 illustratedin FIG. 12A shows a red light. Therefore, when an incident or accidentoccurred, the traffic signal in the visual field direction RNG1 is a redlight, and the vehicle image ESC11 shows that the vehicle was intendedto run while ignoring the condition of the traffic signal. At the top ofthe captured video, the date and time information of the event such asan incident or accident, input to the retrieval terminal 90 may bedisplayed. In the following descriptions, the same applies.

When receiving the operator's operation to change the visual fielddirection RNG1, the retrieval terminal 90 transmits an analysisintroduction to the recording server 50 or the cloud server 70 toanalyze a situation indicated by the traffic situation indicator (forexample, traffic signal) in a visual field direction RNG2, usinginformation on the changed visual field direction RNG2 as a retrievalkey. The center direction of the visual field direction RNG2 correspondsto a direction obtained by rotating the center direction of the visualfield direction RNG1 by 90 degrees. Thus, as illustrated in FIG. 12B,the visual field direction to which the operator pays attention, thatis, the direction that the operator wants to check a traffic situationin case of an event such as an incident or accident is changed from acamera C11 for capturing the visual field direction RNG1 to a camera C12for capturing the visual field direction RNG2. The retrieval terminal 90correlates a traffic signal image SGN2 showing the traffic situationindicator (for example, traffic signal) in the visual field directionRNG2 with a vehicle image (not illustrated) showing a vehicle in thevisual field direction RNG2, using the analysis result transmitted fromthe recording server 50 or the cloud server 70, and overlaps the trafficsignal image SGN2 and the vehicle image with the captured video todisplay on the output unit 94. The traffic signal image SGN2 shows ablue light. Therefore, when the incident or accident occurred, thetraffic signal in the visual field direction RNG2 is a blue light, whichshows that the vehicle (not illustrated) was intended to run whileignoring the condition of the traffic signal.

In the above-described investigation assist system 100 according to thefifth embodiment, the retrieval terminal 90 is connected so as tocommunicate with the recording server 50 or the cloud server 70 whichacquires videos captured by the cameras installed at each of theplurality of intersections. The retrieval terminal 90 acquires capturedvideos of one or more cameras installed at the intersection where theevent such as an incident or accident occurred among the plurality ofintersections from the recording server 50 or the cloud server 70through the communication unit 93 (an example of an acquisition unit),and displays the acquired videos on the output unit 94. The retrievalterminal 90 receives the date and time information of the event and anoperation of inputting the visual field direction for the intersectionwhere the event occurred, through the operation unit 91. The retrievalterminal 90 transmits an analysis request to the recording server 50 orthe cloud server 70 through the communication unit 93 (an example of arequest unit) to analyze the situation indicated by the trafficsituation indicator (for example, traffic signal) in the visual fielddirection, using the captured video corresponding to the date and timeinformation of the camera. The retrieval terminal 90 overlaps theanalysis result for the situation indicated by the traffic situationindicator, transmitted from the recording server 50 or the cloud server70, with the captured video corresponding to the date and timeinformation of the camera, and displays the resultant video on theoutput unit 94.

Accordingly, when an incident or accident occurred in an intersectionwhere many people or vehicles come and go, the retrieval terminal 90 canvisually show a traffic situation around the vehicle having caused theincident or accident at a point of time that the incident or accidentoccurred, and assist the operator to correctly record the situationduring the incident or accident.

The retrieval terminal 90 transmits an analysis instruction to therecording server 50 or the cloud server 70 through the communicationunit 93 (an example of the request unit), such that the recording server50 or the cloud server 70 analyzes the situation indicated by thetraffic situation indicator in the visual field direction changed by theoperation of changing the visual field direction. Accordingly, even whenthe visual field direction to which the operator pays attention waschanged, the retrieval terminal 90 can enable the operator to easilycheck the situation indicated by the traffic situation indicator for thechanged visual field direction, through a simple operation ofdesignating the visual field direction of interest.

The traffic situation indicator is a traffic signal, for example.Therefore, the operator can easily check whether there was a vehiclehaving ignored the traffic signal at the intersection during theincident or accident, and track the vehicle.

The traffic situation indicator is a sign, for example. Therefore, theoperator can easily check whether there was a vehicle having ignored thesign at the intersection during the incident or accident, and track thevehicle.

Background to Sixth Embodiment

In JP-A-2007-174016, the camera image information captured by theplurality of cameras is displayed on the display device of the terminaldevice mounted on the vehicle. Therefore, a user (for example, driver)can check real-time images at the locations where the respective camerasare disposed. However, in JP-A-2007-174016, it is not considered tonarrow down an escape route of a vehicle having caused an event such asan incident or accident, when the event occurred in a traveling route ofthe vehicle (for example, an intersection where many people and vehiclescome and go). For example, when an operator can narrow down anintersection where a getaway vehicle will appear after passing throughthe intersection where the event occurred in the case where a pluralityof intersections are located adjacent to each other, it is possible toreduce labor and time required for retrieving the escape route of thegetaway vehicle. For this reason, even though the technique described inJP-A-2007-174016 is used when the above-described event such as anincident or accident occurred, the operator cannot easily track theescape route of the getaway vehicle, and has difficulties in finding thesuspect or criminal of the incident or accident in early stage.

Therefore, the following sixth embodiment provides an investigationassist system and an investigation assist method, which can efficientlyreduce labor and time required for narrowing down a getaway vehiclehaving run away from an intersection, and assist finding the suspect orcriminal of the incident or accident in early stage, when an incident oraccident occurred in an intersection where many people and vehicle comeand go.

Sixth Embodiment

Since the investigation assist system 100 according to the sixthembodiment has the same configuration as the investigation assist system100 according to the first embodiment, the same components arerepresented by like reference numerals, the detailed descriptionsthereof are simplified or omitted, and the following descriptions willbe focused on different components.

FIG. 13 is a diagram for describing an operation of narrowing down agetaway vehicle through the recording server 50 or the cloud server 70according to the sixth embodiment. FIG. 14 is a flowchart illustratingthe procedure of the operation of narrowing down a getaway vehiclethrough the recording server 50 or the cloud server 70 according to thesixth embodiment. The procedure of the operation of narrowing down agetaway vehicle in FIG. 14 is performed by the recording server 50 orthe cloud server 70 at step Sc15 of the investigation scenario in FIG.6. In each of a plurality of intersections including intersections INTC1to INCT8 illustrated in FIG. 13, one or more cameras (for example, onecamera or four cameras) are installed.

Suppose that an incident occurred at a point Ps1 of the intersectionINTC1 as illustrated in FIG. 13. When receiving an operator's operationof inputting the date and time information and intersection informationof an event such as an incident or accident, the retrieval terminal 90transmits a retrieval request (retrieval instruction) to the recordingserver 50 or the cloud server 70, the retrieval request containing thedate and time information and the intersection information as aretrieval key. Based on the intersection information (information on thepoint Ps1 of the intersection INTC1) and the date and time information,the recording server 50 or the cloud server 70 extracts captured videosof the corresponding cameras, and transmit the extracted videos todisplay on the output unit 94 of the retrieval terminal 90. Theretrieval terminal 90 sequentially displays the captured video datatransmitted from the recording server 50 or the cloud server 70 on theoutput unit 94.

In the sixth embodiment, the recording server 50 or the cloud server 70extracts and acquires captured video data of cameras since the date andtime of the accident from the storage unit 52 or 72, the camerasinstalled in each of the intersections INTC2 to INTC5 adjacent to theintersection INTC1, and transmits the captured video data to display onthe output unit 94 of the retrieval terminal 90. The retrieval terminal90 sequentially displays the captured video data transmitted from therecording server 50 or the cloud server 70 on the output unit 94.Furthermore, the recording server 50 or the cloud server 70 analyzes thecaptured video data since the date and time of the accident, extractedfrom the respective cameras, and thus determines a presence of thevehicle having caused the accident at the point Ps1 or tracks thevehicle.

Whenever the getaway vehicle is detected at any one intersection (forexample, intersection INTC5) among the intersections INTC2 to INCT5through the above-described video analysis, the recording server 50 orthe cloud server 70 extracts feature information on the getaway vehicle,and updates the feature information of the getaway vehicle. Similarly,the recording server 50 or the cloud server 70 extracts and acquirescaptured video data of cameras since the date and time of the accidentfrom the storage units 52 and 72, the cameras being installed in each ofthe intersections INTC6 to INTC8 and INTC1 adjacent to the intersectionINTC5, and repeatedly transmits the captured video data to display onthe output unit 94 of the retrieval terminal 90. The retrieval terminal90 sequentially displays the captured video data transmitted from therecording server 50 or the cloud server 70 on the output unit 94. Therecording server 50 or the cloud server 70 analyzes the captured videodata since the date and time of the accident, extracted from therespective cameras, and thus determines the presence of the getawayvehicle detected at the intersection INTC5 or tracks the vehicle. Therecording server 50 or the cloud server 70 repeats the process.

Based on the position information of the intersection where the runawaywas detected, the recording server 50 or the cloud server 70 may overlapthe information on the detected route (escape route) of the getawayvehicle with road map information recorded (stored) in the storage unit52 or 72, and transmit the road map information to display on the outputunit 94 of the retrieval terminal 90. Furthermore, the recording server50 or the cloud server 70 may derive a score indicating the possibilitythat each of the vehicles will be the getaway vehicle (refer to thefourth embodiment), and transmit the captured video data of thecorresponding vehicles to the retrieval terminal 90 such that thecaptured video data are displayed on the output unit 94 in descendingorder of score.

When the captured videos of the cameras installed at each of theplurality of intersections are displayed on the retrieval terminal 90,the recording server 50 or the cloud server 70 may generate the capturedvideo data as follows. Under the supposition that a vehicle consideredas a getaway vehicle is being captured by a plurality of cameras or aplurality of frames, the recording server 50 or the cloud server 70extracts data of a video in which a vehicle designated by the operatoris captured, using the feature information (for example, type or shapeof the vehicle) of the designated vehicle in the captured videodisplayed on the output unit 94. The recording server 50 or the cloudserver 70 may generate display data by generating still images of thegetaway vehicle and sorting the still images in order of time axes atwhich the images were captured, and display the generated display dataon the retrieval terminal 90. Furthermore, the recording server 50 orthe cloud server 70 may generate display data by generating still imagesof the getaway vehicle and sorting the still images in order of degreeof definition, and display the generated display data on the retrievalterminal 90. Moreover, the recording server 50 or the cloud server 70may generate display data by generating still images of the getawayvehicle and classifying and sorting the still images depending on thedirection of the vehicle (for example, front, read or side) in therespective still images, and display the generated display data on theretrieval terminal 90. In this case, the recording server 50 or thecloud server 70 may transmit the still images and display data to theretrieval terminal 90, the display data capable of displayinginformation indicating the direction of the camera.

In FIG. 14, the recording server 50 or the cloud server 70 calculates aperiod of time from the time at which the report was received or thegetaway vehicle was finally found (detected) to the current time,through the video retrieval unit 53 or 73, at step S41. The recordingserver 50 or the cloud server 70 retrieves a captured video of a camerathrough the video retrieval unit 53 or 73, the camera being located at adistance that the vehicle can normally move within the time calculatedat step S41, from the camera connected by the report or the camera bywhich the getaway vehicle was finally found (detected), at step S42.

When the video retrieval unit 53 or 73 specifies the cameracorresponding to a retrieval target at step S42, the video retrievalunit 53 or 73 does not use a physical straight distance from theoccurrence point of the accident (accident occurrence point notified bythe report) or the point where the getaway vehicle was finally detected,but uses a connection relation between intersections located within adistance from the point and included in the road map informationrecorded (stored) in the storage unit 52 or 72. The connection relationbetween the intersections may include the distance between theintersections or the average speed of the vehicle moving between theintersections. For example, when the connection distance of a road fromthe intersection INTC1 to the intersection INTC5 is 1 km, the averagespeed of the vehicle from the intersection INTC1 to the intersectionINTC5 is 20 km/h, and the time calculated at step S41 is five minutes,the intersection INTC5 is included in the camera of the retrieval targetat step S42. Furthermore, when the connection distance of a road fromthe intersection INTC1 to the intersection INTC6 is 2 km, the averagespeed of the vehicle from the intersection INTC1 to the intersectionINTC6 is 10 km/h, and the time calculated at step S41 is five minutes,the intersection INTC6 is excluded from the camera of the retrievaltarget at step S42.

When the getaway vehicle was found (detected) as the retrieval result ofstep S42 (Yes at step S43), the recording server 50 or the cloud server70 updates the detection time information of the vehicle detected atstep S43 and the information of the location where the vehicle wasdetected, and records (stores) the updated information in the storageunit 52 or 72, at step S44. After step S44, the process of the recordingserver 50 or the cloud server 70 is returned to step S41 and thenrepeated.

When the recording server 50 or the cloud server 70 could not find(detect) the getaway vehicle (No at step S43), the recording server 50or the cloud server 70 performs the process of step S41. That is, therecording server 50 or the cloud server 70 specifies a camera capable ofdetecting the getaway vehicle again. Whenever returning to step S41after step S44, the recording server 50 or the cloud server 70 may waitfor a predetermined waiting time (for example, 5 to 30 seconds). Inother words, since the retrieval area is not widened even though theloop is precisely performed, the recording server 50 or the cloud server70 performs retrieving again when a new intersection is included in theretrieval area or the number of passing vehicles increases after apredetermined time has elapsed. That is, whenever the process of stepS44 is performed, the recording server 50 or the cloud server 70 mayperform the process of step S41 after a predetermined time (for example,5 to 30 seconds) has elapsed since the process of step S44. Accordingly,the recording server 50 or the cloud server 70 does not subdivide theretrieval area of the getaway vehicle more than needed, but can increasethe number of camera candidates included in the retrieval area bysetting the predetermined time to a grace period, thereby improving theretrieval precision for the getaway vehicle.

In the investigation assist system 100 according to the sixthembodiment, the recording server 50 or the cloud server 70 correlatesthe captured videos of the camera with the road map informationincluding the plurality of intersections, the camera information, andthe intersection information, and records the resultant information inthe storage unit 52 or 72. As the date and time information and theintersection information of the event such as an incident or accidentand the feature information of the vehicle having caused the event areinput to the retrieval terminal 90, the recording server 50 or the cloudserver 70 retrieves the vehicle through the video retrieval unit 53 or73 (an example of the retrieval unit), using captured videos of thecameras at one or more intersections within a predetermined distancerange from the intersection where the event occurred, the capturedvideos corresponding to the date and time information. The recordingserver 50 or the cloud server 70 updates the detection date and timeinformation of the vehicle from the reception date and time informationof the report indicating the occurrence of the event to the detectiondate and time information of the vehicle based on retrieval, updates thedetection point information of the vehicle from the information on theintersection where the event occurred from information on anotherintersection where the vehicle was detected through retrieval, throughthe video retrieval unit 53 or 73, and stores the updated information inthe storage unit 52 or 72.

Accordingly, when an incident or accident occurred at an intersectionwhere many people or vehicles come and go, the recording server 50 orthe cloud server 70 can efficiently reduce labor and time required fornarrowing down a vehicle having run away from the intersection, andefficiently assist finding the suspect or criminal of the accident orincident in early stage.

The recording server 50 or the cloud server 70 analyzes the featureinformation of the vehicle detected by retrieval (for example, getawayvehicle) through the video retrieval unit 53 or 73, updates the featureinformation of the getaway vehicle by adding the analyzed featureinformation to the feature information of the vehicle, input by theoperator, and stores the updated information in the storage unit 52 or72. Therefore, since the feature information of the getaway vehicle canbe additionally increased, the recording server 50 or the cloud server70 can improve the reliability of the specifying of the getaway vehiclethrough the analysis of the captured video data, and display thecaptured video data of the getaway vehicle on the output unit 94 of theretrieval terminal 90, such that the operator can visually check thegetaway vehicle.

The recording server 50 or the cloud server 70 extracts a camera of anintersection, used for retrieving, through the video retrieval unit 53or 73, while a distance that the vehicle can move during a timedifference between the current time information and the detection dateand time information of the vehicle detected by retrieving is set to apredetermined distance. Therefore, the recording server 50 or the cloudserver 70 not only considers a simple straight distance, but alsoconsiders a connection relation between roads or a time required formovement of the getaway vehicle. Thus, the recording server 50 or thecloud server 70 can easily specify a camera which can easily track thegetaway vehicle.

The recording server 50 or the cloud server 70 repeatedly performs thevehicle retrieval through the video retrieval unit 53 or 73, usingcaptured videos of the cameras at one or more intersections within apredetermined distance range from another intersection where the vehiclewas detected by the retrieval, the captured video corresponding to thedetection date and time information of the vehicle. Accordingly, therecording server 50 or the cloud server 70 can continuously track thegetaway vehicle, and reduce labor and time required for retrieving thegetaway vehicle.

Background to Seventh Embodiment

In JP-A-2007-174016, the camera image information captured by theplurality of cameras is displayed on the display device of the terminaldevice mounted on the vehicle. Therefore, a user (for example, driver)can check real-time images at the locations where the respective camerasare disposed. However, JP-A-2007-174016 does not consider narrowing downan escape route of a vehicle having caused an event such as an incidentor accident, when the event occurred around a traveling route of thevehicle (for example, an intersection where many people and vehiclescome and go). For this reason, even though the technique described inJP-A-2007-174016 is used when the above-described event such as anincident or accident occurred, the operator cannot track the escaperoute of the getaway vehicle, and has difficulties in finding thesuspect or criminal of the incident or accident in early stage.

Therefore, the following seventh embodiment provides an investigationassist device, an investigation assist method and an investigationassist system, which can efficiently narrow down an escape route of agetaway vehicle, and assist finding the suspect or criminal of anincident or accident in early stage, when the incident or accidentoccurred around an intersection where many people and vehicle come andgo.

Seventh Embodiment

Since the investigation assist system 100 according to the seventhembodiment has the same configuration as the investigation assist system100 according to the first embodiment, the same components arerepresented by like reference numerals, the detailed descriptionsthereof are simplified or omitted, and the following descriptions willbe focused on different components.

FIG. 15 is a diagram for describing an operation of narrowing down agetaway vehicle through the recording server 100 according to theseventh embodiment. FIG. 16 is a diagram for describing the operation ofnarrowing down a getaway vehicle through the recording server 100according to the seventh embodiment. FIG. 17 illustrates a screen fordisplaying an escape route of a getaway vehicle. FIG. 18 is a diagramfor describing the procedure of the operation of narrowing down agetaway vehicle through the recording server 100 according to theseventh embodiment. The procedure of the operation of narrowing down agetaway vehicle in FIGS. 15, 16 and 18 is performed by the recordingserver 50 or the cloud server 70 at step Sc15 of the investigationscenario illustrated in FIG. 6.

At step i1 of FIG. 15, suppose that there was a report saying that anaccident occurred at a point Ps2 at time T0, the type of a vehiclehaving caused the accident is a car, the color of the vehicle is yellow,and the last digit of the number plate is 3. An operator in a policestation inputs the various pieces of report information received throughthe report as data, generates a retrieval request (retrievalinstruction) containing the data as a retrieval key, and transmits theretrieval request to the recording server 50 or the cloud server 70.

At step i2, the recording server 50 or the cloud server 70 retrievesdata of captured videos MOVE and MOV7 of cameras at a point of time thatthe getaway vehicle is likely to pass through one or more intersectionsadjacent to the point Ps2 corresponding to the reported accident site,based on the retrieval request (retrieval instruction) transmitted fromthe retrieval terminal 90. The recording server 50 or the cloud server70 extracts image lists CND1 and CND2 of vehicles detected in capturedvideos for X minutes before and after the point of time that the getawayvehicle is likely to pass through the intersection (time T0+α), andtransmits a retrieval result containing the extracted vehicle imagelists and the captured video data such that the retrieval result isdisplayed on the output unit 94 of the retrieval terminal 90. Theretrieval terminal 90 displays display screens WD5 and WD6 of thecorresponding captured videos MOV6 and MOV7 transmitted from therecording server 50 or the cloud server 70, the vehicle image lists CND1and CMD2, and reproduction control bars TL1 and TL2 of the capturedvideos MOV6 and MOV7 on the output unit 94. The reproduction controlbars TL1 and TL2 are used to control the reproduction times of thecaptured videos MOV6 and MOV7. When a time required from the point oftime that the getaway vehicle was detected (for example, the time T0) tothe point of time that the getaway vehicle passes through the nextintersection is a, the vehicle image lists show the vehicle detected inthe captured videos during a predetermined time width (time 2×)including the time T0+α. Therefore, it is possible to say that thegetaway vehicle is highly likely to be included in the vehicle imagelists. The operator having input the report information selects theimage of any one vehicle which is highly likely to be the getawayvehicle. According to the selection of the operator, the retrievalterminal 90 applies a color frame SLT1 onto the selected vehicle, andtransmits the identification information of the selected vehicle to therecording server 50 or the cloud server 70. In the descriptions of FIGS.15 and 16, the point of time that the vehicle having caused the accidentat the point Ps2 was detected through the report is set to T0, and thepoint of time that the vehicle was detected at an intersection (3)adjacent to a hexagonal intersection (2) is set to T1.

FIG. 16 illustrates road map information at the top thereof, the roadmap information showing the intersection where the vehicle correspondingto the image of the vehicle candidate selected at step i2 was detected(specifically, hexagonal intersection (2)) and two intersections(specifically, intersections (1) and (3)) adjacent to the intersection.At step i3, the recording server 50 or the cloud server 70 receives theidentification information of the vehicle from the retrieval terminal90, and retrieves data of captured videos MOV8 and MOV9 of cameras at apoint of time that the getaway vehicle is likely to pass through one ormore intersections adjacent to the intersection where the vehicle wasdetected (specifically, hexagonal intersection (2)), based on thereceived identification information. Furthermore, the recording server50 or the cloud server 70 extracts image lists CND1 and CND3 of vehiclesdetected in the captured videos for X minutes before and after the pointof time that the getaway vehicle is likely to pass (time T1+α), andtransmits a retrieval result containing the extracted vehicle imagelists and the captured video data such that the retrieval result isdisplayed on the output unit 94 of the retrieval terminal 90. Theretrieval terminal 90 displays display screens WD5 and WD6 of thecorresponding captured videos MOV8 and MOV9 transmitted from therecording server 50 or the cloud server 70, the vehicle image lists CND1and CND3, and reproduction control bars TL1 and TL2 of the capturedvideos MOV8 and MOV9 on the output unit 94. The reproduction controlbars TL1 and TL2 are used to control the reproduction times of thecaptured videos MOV8 and MOV9. The vehicle with the frame SLT1, selectedby the operator at step Si2, was not detected at the intersection (1)and not detected in the captured videos for the X minutes before andafter, including the time T1+α. Therefore, the image of the vehicle isnot displayed in the list CND1 at step i3. However, the list CND2 showsthe vehicle with the frame SLT1, selected by the operator at step Si2,and one more vehicle. When the time required from the point of time thatthe getaway vehicle was detected (for example, time T1) to the point oftime that the getaway vehicle passes through the next intersection is a,the vehicle image lists show the vehicles detected in the capturedvideos during a predetermined time width (time 2×) including the timeT1+α. Therefore, it is possible to say that the getaway vehicle ishighly likely to be included in the vehicle image lists. The operatorhaving input the report information selects the image of any one vehiclewhich is highly likely to be the getaway vehicle. The retrieval terminal90 applies the colored frame SLT1 onto the selected vehicle, accordingto the selection. Even after step Si3, the same process as steps i2 andi3 is repeated.

By repeating the process of steps i2 and i3 and the process after stepi3, the recording server 50 or the cloud server 70 may generate data inwhich an escape route Rot1 of the getaway vehicle is overlapped with theroad map information as illustrated in FIG. 17, and transmit the data todisplay on the output unit 94 of the retrieval terminal 90. Theretrieval terminal 90 displays the road map information received fromthe recording server 50 and the cloud server 70 on the output unit 94.Accordingly, the operator can visually and intuitively recognize theescape route of the getaway vehicle from the report about the occurrenceof the accident, and efficiently provide information to theinvestigation headquarter which supervises the suitable initialinvestigation of a policeman or the like. Furthermore, informationindicating the escape direction (for example, an arrow or timeinformation when the vehicle passes through each intersection) may beapplied to the displayed escape route Rot1. For example, in the presentembodiment, an arrow indicating a moving direction from the intersection(2) to the intersection (3) may be displayed on the escape route Rot1.

In FIG. 18, when the feature information of a vehicle having caused anaccident is input through an operation input by the operator based onthe report about the occurrence of the accident at step S51 and the dateand time information and the site information of the accident are inputat step S52, the retrieval terminal 90 transmits a retrieval request(retrieval instruction) containing the input information as a retrievalkey to the recording server 50 or the cloud server 70. Based on theretrieval request (retrieval instruction) transmitted from the retrievalterminal 90, the recording server 50 or the cloud server 70 extracts(specifies) cameras of one or more intersections adjacent to thereported location or the location where the getaway vehicle was detectedimmediately before, through the video retrieval unit 53 or 73, at stepS53. At step S53, the recording server 50 or the cloud server 70specifies the cameras corresponding to the extraction target, using theroad map information recorded (stored) in the storage unit 52 or 72 orinformation obtained by correlating the camera information and theintersection information with each other.

After step S53, the recording server 50 or the cloud server 70 sets aretrieval target period for retrieving captured videos of the camerasthrough the video retrieval unit 53 or 73, using the information on thepoint of time that the report was received (the date and timeinformation of the event) or the point of time that the getaway vehiclewas detected immediately before (in FIG. 18, set to time T0 forconvenience), at step S54. The retrieval target period may include theperiod of the X minutes before and after, including the timeT0+αdescribed with reference to FIG. 15, or the period of the X minutesbefore and after, including the time T1+αdescribed with reference toFIG. 16. Here, a may be set to a fixed value or a value obtained bydividing a distance by the average moving speed of the vehicle, thedistance ranging from the point at which the getaway vehicle has beenmost recently detected to an intersection where the getaway vehicle ishighly likely to be detected next time. That is, a represents the timefrom the point of time that the accident occurred or the getaway vehiclehas been recently detected to the point of time that the same getawayvehicle is detected.

The recording server 50 or the cloud server 70 retrieves a vehiclehaving the feature information input at step S51 through the videoretrieval unit 53 or 73, using captured video data of the camerasextracted at step S53 during the retrieval target period set at step S54(step S55).

When the vehicle retrieval is performed for the first time or themaximum similarity (refer to the score in the fourth embodiment) as theretrieval result is equal to or less than a predetermined thresholdvalue (Yes at step S56), the recording server 50 or the cloud server 70extracts an image list of a plurality of vehicles detected in thecaptured videos during the retrieval target period set at step S54, asthe retrieval result of step S55, and transmits a retrieval resultcontaining the extracted vehicle image list and the captured video data,such that the retrieval result is displayed on the output unit 94 of theretrieval terminal 90, at step S57. According to a selection operationof the operator at step S58, the retrieval terminal 90 applies a colorframe onto the selected vehicle, and transmits the identificationinformation of the selected vehicle to the recording server 50 or thecloud server 70. The recording server 50 or the cloud server 70 analyzesthe feature information of the vehicle selected by the selectionoperation of the operator through the video retrieval unit 53 or 73,updates the feature information of the vehicle (getaway vehicle), inputat step S51, by adding the analysis result to the feature information,and records (stores) the updated information in the storage unit 52 or72, at step S59. Furthermore, the recording server 50 or the cloudserver 70 updates the information of the time and location that thevehicle selected at step S58 was detected, in order to set theinformation of the time and location to the latest information, at stepS61. After step S61, the process of the recording server 50 or the cloudserver 70 returns to step S53, and the process from step S53 to step S61is repeated.

On the other hand, when the vehicle retrieval is not the first retrievaloperation and the maximum similarity as the retrieval result (refer tothe score in the fourth embodiment) is larger than the predeterminedthreshold value (No at step S56), it is considered that the vehicleselected at step S58 is the getaway vehicle and the precision of theprocess of retrieving the getaway vehicle (refer to step S55) is high.In this case, the recording server 50 or the cloud server 70continuously transmits images of the getaway vehicle selected at stepS58 as a formal process, such that the images are displayed on theoutput unit 94 of the retrieval terminal 90, at step S60. After stepS60, the recording server 50 or the cloud server 70 updates theinformation of the time and location that the vehicle which iscontinuously selected even at step S60 was detected, in order to set theinformation of the time and location to the latest information, at stepS61. After step S61, the process of the recording server 50 or the cloudserver 70 returns to step S53, and the process from step S53 to step S61is repeated.

In the investigation system 100 according to the seventh embodiment, therecording server 50 or the cloud server 70 is connected to the camerasinstalled at each of the plurality of intersections so as to communicatewith the cameras. The recording server 50 or the cloud server 70correlates the captured videos of the cameras with the camerainformation, the intersection information and the road map informationincluding the plurality of intersections, and records the resultant datain the storage unit 52 or 72. According to an information inputcontaining information on a first point (for example, the point Ps1)where an event occurred and the feature information of the vehiclehaving caused the event, the recording server 50 or the cloud server 70extracts camera information of intersections adjacent to the first pointthrough the video retrieval unit 53 or 73. The recording server 50 orthe cloud server 70 retrieves a vehicle satisfying the featureinformation of the vehicle through the video retrieval unit 53 or 73,using captured videos of cameras specified by the extracted camerainformation. The recording server 50 or the cloud server 70 analyzes thefeature information of the retrieved vehicle through the video retrievalunit 53 or 73. The recording server 50 or the cloud server 70 updatesthe feature information of the vehicle, input through the informationinput, through the video retrieval unit 53 or 73 by adding the analyzedfeature information to the feature information, and stores the updatedinformation in the storage unit 52 or 72. The recording server 50 or thecloud server 70 updates the detected point information of the vehiclefrom the information on the first point (for example, the point Ps1)into the detection point information of the retrieved vehicle throughthe video retrieval unit 53 or 73, and stores the updated information inthe storage unit 52 or 72.

Accordingly, when an incident or accident occurred at an intersectionwhere many people or vehicles come and go, the recording server 50 orthe cloud server 70 can efficiently narrow down an escape route of agetaway vehicle, and efficiently assist finding the suspect or criminalof the accident or incident in early stage.

The recording server 50 or the cloud server 70 retrieves a plurality ofvehicles satisfying the feature information of the vehicle, transmitsimages of the retrieved vehicles to display on the retrieval terminal90, and analyzes the feature information of any one vehicle selected bythe retrieval terminal 90 among the displayed vehicle image. Therecording server 50 or the cloud server 70 updates the detection pointinformation of the vehicle into the detection point information of anyone vehicle selected by the retrieval terminal 90. Accordingly, as theoperator receiving the report and having the report contents in his headdirectly selects a vehicle which is highly likely to be the getawayvehicle, the recording server 50 or the cloud server 70 can update thefeature information of the getaway vehicle, narrow down captured videosof the getaway vehicle with high precision, and provide the capturedvideos to display on the retrieval terminal 90.

Whenever the detection point information of the vehicle is updated, therecording server 50 or the cloud server 70 extracts camera informationof intersections adjacent to the detection point specified by theupdated detection point information of the vehicle. Accordingly, sincethe getaway vehicle shows the characteristic that the getaway vehicle ishighly likely to pass through any one intersection having camerasinstalled therein, the recording server 50 or the cloud server 70 canupdate the detection point information of the getaway vehicle wheneverthe getaway vehicle is detected at an intersection, and easily narrowdown the escape route of the getaway vehicle.

The recording server 50 or the cloud server 70 sets a predeterminedperiod as a retrieval target period for captured videos of the camerasthrough the video retrieval unit 53 or 73, the predetermined periodcontaining the time of the report about the occurrence of the event orthe detection time of the vehicle. Accordingly, since the predeterminedperiod required until the getaway vehicle is detected in a capturedvideo of a camera after the getaway vehicle was detected is validly set,the recording server 50 or the cloud server 70 can improve the detectionprecision for the getaway vehicle.

Although the various embodiments have been described with reference tothe drawings, the present disclosure is not limited to the embodiments.It is obvious to those skilled in the art that various changes andmodifications can be made within the category described in claims, andbelong to the scope of the present disclosure. Moreover, the componentsin the various embodiments may be arbitrarily combined without departingthe scope of the present disclosure.

The present disclosure is usefully applied as the investigation assistdevice, the investigation assist method and the investigation assistsystem, which can effectively narrow down escape routes of a getawayvehicle, and assist finding the suspect or criminal of an incident oraccident in early stage, when the incident or accident occurred aroundan intersection where many people or vehicles come and go.

What is claimed is:
 1. An investigation assist system comprising: aninvestigation assist device that acquires captured video of camerasinstalled at a plurality of intersections respectively; and an operatorterminal that is connected to the investigation assist device so as tocommunicate with the investigation assist device; wherein theinvestigation assist device detects an occurrence of an event at a firstpoint based on the acquired captured video of the camera, and transmitsthe captured video of the camera to the operator terminal, the capturedvideo being acquired at a time when the event is detected, wherein theoperator terminal records the captured video of the camera transmittedfrom the investigation assist device, wherein the operator terminalreceives a report indicating the occurrence of the event, and wherein,repeatedly at least twice: the operator terminal generates, based on thereport indicating the occurrence of the event, a retrieval key, andtransmits a retrieval request including the retrieval key to theinvestigation assist device, the investigation assist device receivesthe retrieval request, extracts a captured video satisfying theretrieval key transmitted from the operator terminal, and transmits theextracted video to the operator terminal, and the operator terminaldisplays the captured video transmitted from the investigation assistdevice on a display.
 2. The investigation assist system according toclaim 1, wherein the operator terminal transmits the retrieval requestto the investigation assist device every time the retrieval key based onthe report is input, and wherein the investigation assist deviceextracts captured video data satisfying the retrieval key transmittedfrom the operator terminal, and transmits the extracted video data tothe operator terminal.
 3. An investigation assist method using aninvestigation assist device that acquires captured video of camerasinstalled at a plurality of intersections respectively and an operatorterminal that is connected to the investigation assist device so as tocommunicate with the investigation assist device, the investigationassist method comprising: detecting an occurrence of an event at a firstpoint based on the acquired captured video of the camera, transmittingthe captured video of the camera to the operator terminal, the capturedvideo being acquired at a time when the event is detected, recording thecaptured video of the camera transmitted from the investigation assistdevice, and repeatedly at least twice: transmitting a retrieval requestincluding a retrieval key generated based on a report indicating theoccurrence of the event to the investigation assist device, extracting acaptured video satisfying the retrieval key transmitted from theoperator terminal, transmitting the extracted video to the operatorterminal, and displaying the captured video transmitted from theinvestigation assist device on a display.